Systems and methods for machine content generation

ABSTRACT

Computerized systems and methods are disclosed to generate a document by providing a document structure having one or more seed landmark texts therein, each landmark text including a milestone overview text and a plurality of component texts; from the milestone overview text, generating one or more computer-generated text suggestions to supplement the milestone overview text; combining the milestone overview text with each component text and generating one or more computer-generated component text suggestions; and creating the document by combining the milestone overview, the one or more computer-generated text suggestions, and each component text with corresponding one or more computer-generated component text suggestions.

The present invention relates to computer assisted or automated contentgeneration.

BACKGROUND

Writing well is a must have skill for professionals such as poets andauthors, and for corporate employees, writing one of those skills thatcan help a person rise above in her career. For novelists and writers,getting started can be hard. A writer is not someone who thinksobsessively about writing, or talks about it, or plans it, or dissectsher writing. The terror of the white page is real for most. When suchroadblocks occur, the writer can lose valuable time in completing a bookon time and on-budget. Video content creators face similar issues whenpitching new movie concepts to funders. FIG. 1A shows a conventionaloutlining method for books, while FIG. 1B shows a conventional outliningmethod for video/movie content. In these examples, FIG. 1A showsexemplary plans for the Harry Potter books while FIG. 1B shows a samplewell-known science fiction movie storyboard.

Generating natural language from machine representation systems is acommon and increasingly important function. Existing natural languagegeneration (NLG) systems, such as translators, summarizers, dialoggenerators, etc., while common, cannot produce variable output based onuser-desired tunable specifications. Additionally, such existing systemscannot take input in the form of a variable form of text and a variableset of specifications and output a transformed version of the input textaccording to the specifications. Further, such existing systems aregenerally not readily extendable. US Application 20200034432 mentionsgenerating tunable stylized text (such as, for example, one or moresentences) by transforming received user text input and one or moreuser-originated stylistic parameters (directed to polarity of subjectiveopinion, such as sentiments, valence, emotions, formal, business,readability, etc.) in vector form, using unsupervised natural languageprocessing (NLP) systems such as rule-based and/or machinelearning-based classifiers and/or regressors, metric computation systemsas style scorers, etc.

SUMMARY

The AI architecture herein can be used for communication, for example,to generate long text or video using the neural network architectures.In one aspect for AI content generation, computerized systems andmethods are disclosed to generate a document by providing a documentstructure having one or more seed landmark texts therein, each landmarktext including a milestone overview text and a plurality of componenttexts; from the milestone overview text, generating one or morecomputer-generated text suggestions to supplement the milestone overviewtext; combining the milestone overview text with each component text andgenerating one or more computer-generated component text suggestions;and creating the document by combining the milestone overview, the oneor more computer-generated text suggestions, and each component textwith corresponding one or more computer-generated component textsuggestions.

In yet another aspect, a method to generate content with a plurality ofimages or video includes providing a multimedia structure having one ormore seed landmark images therein, each landmark image including amilestone overview text and a plurality of component texts; from themilestone overview text, generating one or more computer-generated imagesuggestions to supplement the landmark image; combining the milestoneoverview text with each component text and generating one or morecomputer-generated component image suggestions; and creating the contentby combining the landmark image and the one or more computer-generatedimage suggestions.

In a further aspect, a method provides a chatbot trained with contextsensitive data whose response is biased during runtime with highlycustomized responses and with realistic human like response ispresented.

In yet another aspect, a chatbot serves in place of human agents toprovide answers for customers. The bot detects user emotions and if itdetects charged emotions, get help from the best matching agent to helpthe customer.

In yet a further aspect, a web site content generator renders AI contentthat is SEO optimized. The text includes ontology or semantic tags toaid a search engine in locating best matching responses that are innatural language.

Implementations of the above aspects may include one or more of thefollowing additions to the above aspect:

2. the document structure comprises an outline, wherein each landmarktext comprises a chapter overview, and wherein the component textscomprise a chapter outline.

3. the document comprises a fiction work, a non-fiction work, a computerreadable code, a machine specification, or a mechanical description.

4. the document structure comprises one or more figures, wherein eachfigure comprises a brief description of the drawing, a figuredescription overview, and a detailed description for the figure withcomponent texts corresponding to items in the figure.

5. biasing neural network weights with the milestone overview text whengenerating a context-sensitive component text suggestion.

6. the combining further comprises combining a title and a backgroundtext with the one or more seed landmark texts and providing the combinedtitle, background, and seed landmark texts to a learning machine tosynthesize artificial-intelligence-generated text.

7. extracting one or more references from a figure and annotating theone or more references with text; and forming one or moreartificial-intelligence-generated reference text suggestions.

8. performing grammar analysis and suggesting grammar correction andediting the document for conciseness.

9. applying a transformer with an encoder that reads the text input anda decoder that produces a prediction for the text.

10. the transformer comprises a generative pre-trained transformer(GPT).

11. applying GPT (Generative Pre-trained Transformer) model or a BERT(Bidirectional Encoder Representations from Transformers) model togenerate the text.

12. determining when two pieces of text, component, module, code, datastructure, or image perform a similar task and showing the determinedtext, component, module, code, data structure, or image to a user.

13. breaking-down the milestone overview text into one or more alternatecomponents with different component text but capable of performing themilestone overview text based on teachings from prior art documents andshowing the one or more alternate components as aartificial-intelligence-generated design around satisfying the milestoneoverview text, wherein the breaking-down comprises applying anartificial intelligence software to detect similarity of functions.

14. detecting plagiarism in the document by matching the document textto text crawled from the Internet.

15. generating a part list by detecting noun phrases (NPs) in thedocument and corresponding numbers for the NPs.

16. generating a list of claimed elements.

17. generating a list of unclaimed elements.

18. The method of claim 1, wherein the document is part of a portfolioaccessible to one or more licensees.

19. granting rights to the document and/or guiding text generation witha chatbot.

20. generating context-sensitive text by:

-   -   training a learning machine architecture (LMA) a corpus on a        specific domain (such as engineering, medical, chemical,        patent), wherein the architecture can be BERT, GPT, or a        suitable network;    -   using a first text input to retrieve a first set of documents        responsive to the first text input to provide contex;    -   applying the first set of documents as input to the LMA to        generate the context sensitive text.

21. generating long form context-sensitive text by:

-   -   training a learning machine architecture a corpus on a specific        domain (such as engineering, medical, chemical, patent), wherein        the architecture can be BERT, GPT, or a suitable network,        wherein the LMA is trained on 200, 500, or 800 token frames of        data;    -   using a first text input to retrieve a first set of documents        responsive to the first text input to provide contex;    -   applying the first set of documents as input to the LMA to        generate the context sensitive text.

Advantages of the system may include one or more of the following. Thesystem increases communication effectiveness. The system generates goodtechnical writing in a time-saving manner, and the results avoidmisunderstanding and increase workplace efficiency by promoting goodcommunication between engineers and other staff. The system directs thewriting to the intended audience will allow the reader to understand thecontent on the first read, rather than needing to ask for additionaldetails or explanation. By understanding the audience's goal in readingthe document, the system helps the writer to highlight the importantdata, focusing on significant supplementary or background informationand bringing such information to the user to decide. Thus, theinformation needed for a decision, instruction or education take centerstage. The system keeps the information accessible and uses the simplestand most direct language to convey the information with a neutral andprofessional tone. The system helps the users with diagrams orschematics where they add value and increase reader comprehension. Whenused, the diagrams are directly referenced within the text and clearlyexplained in the text. The system provides a Visual and intuitive userinterface with built-in semantic and technical understanding, automaticrelevant passage suggestions. The system reduces the cost of writingdocuments by serving as writing assistants that fill (or inbetween)details based on the abstract. For more technical descriptions whereengineering details are important, the system can expand from anabstract to a full description with clarity. In other applications thatdemand flowery language, the efficiency of human drafters can beimproved significantly when a master drafter generates a summary of themajor points in the article, and the computer fills in the missingdetails, much similar to inbetweening of animation. The user would drawthe keyframes which define the movement, then, hands the scene to ahuman or computer assistant. The assistant does the clean-up and thenecessary inbetweens, or, in large studios, only some breakdowns whichdefine the movement in more detail, before handing down the scene totheir assistant, the inbetweener, who does the rest. The system canadapt the detail resolution or rate to the current scene. Differentscenes components of a story might be animated at different resolutionsor rates to conform to the master drafter's command. The result is asignificant speedup in document generation, while cost is reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

For the purposes of illustrating the invention, there are shown in thedrawing forms which are presently preferred, it being understood,however, that the invention is not limited to the precise arrangementsand instrumentalities shown. It is to be understood that both theforegoing general description and the following detailed description arenot limiting but are intended to provide further explanation. Furtherfeatures and advantages, as well as the structure and operation ofvarious embodiments thereof, are described in detail below withreference to the accompanying drawings. The accompanying drawings whichare incorporated in and constitute part of the specification areincluded to illustrate and provide a further understanding of themethods, systems, and computer program products. Together with thedescription, the drawings explain the principles.

FIG. 1A shows an exemplary storyboard for a video or movie where theplot can be computer generated, human generated, or a combinationthereof.

FIGS. 1B-1E shows exemplary long form content generation user interface.

FIG. 1F shows an exemplary outline user interface.

FIG. 2A-2I shows exemplary long form content generation flowcharts.

FIG. 3A-3C shows exemplary processes to use AI for generating chatbotresponses, for selecting and assisting call center agent in answeringcalls, and for search engine optimization, among others.

FIG. 3D shows an exemplary AI chatbot to respond to infectiousoutbreaks.

FIG. 4A shows top level views of the GPT, BERT, and Transformerarchitectures.

FIG. 4B shows the encoder and decoder stacks of the Transformerarchitecture.

FIG. 4C shows in more detail the encoder and decoder blocks of theTransformer architecture.

FIGS. 4D-4G show additional views of the Transformer architecture forlong-form text generation.

FIG. 4H shows an exemplary adversarial architecture for text or videogeneration.

FIG. 5A-5C show various embodiments for applying the content generationsystem to generate revenues for providing additional resources forschools or educational institutions.

DETAILED DESCRIPTION

The exemplary embodiments consist of major and subsidiary componentsimplemented through a variety of separate and related computer systems.These components may be used either individually or in variety ofcombinations to achieve the objective of providing a new and improvedway to enable content providers to price their specified targetaudience, for purchase or sale, anytime, based on real-time demand orotherwise, and anywhere without limitation of device platform or anassociation with content that may limit the distribution of thatcontent. Further, the disclosed embodiments provide forcommercialization of price optimization mechanisms within organizedelectronic marketplaces where rights to access audience profiles and ordisplay space can be traded, in a primary or secondary market.

It is noted that the invention is not limited to the specificembodiments described herein. Such embodiments are presented herein forillustrative purposes only. Additional embodiments will be apparent topersons skilled in the relevant art(s) based on the teachings containedherein. The section headings used herein are for organizational purposesonly and are not to be construed as limiting the subject matterdescribed.

FIG. 1B shows an exemplary system to generate a document usingcomputer-generated outlines, or alternatively using digitizedhand-crafted outlines or storyboards. Gathering the writer's thoughtswhen writing a novel can be a tricky process, which is why many writersplan their plots. In FIG. 1A, users upload a series of drawings orfigures. In this example, the image uploaded is a handwritten outline ofa book or paper, for example. FIG. 1A shows the storyboard for Star Warsbut can also be a table that shows the planning of the Harry Potterbooks, for example. Storyboards may be comic book illustration of theentire movie, or selected scenes in a movie, including camera angles andthe motion of actors through the sets.

As shown in the example of FIG. 1B, a thumbnail 2, is shown on the leftside and, when clicked, is shown in full size in space 4. The user canannotate major numbers in the image with a text summary in theannotation section 6. The annotations form an outline of the resultingdocument or book. The annotation can have adjustable opaqueness so thatthe annotation can overlay the image. The annotation can be typed in orcan be optically recognized using a learning machine, computer vision(OpenCV), or other suitable machine recognition techniques. The user cantype in brief descriptions of the drawings on the top of space 4, and afew sentences in the detailed description section 10. With that seedinformation, the artificial intelligence software starts suggesting oneor more text paragraphs for the user to adopt or edit/revise and thenadd to the detailed description. Next, the system goes through eachannotation in space 6 and machine-generated additional text suggestionsfor the user to apply to the detailed description.

In FIG. 1B, the user can specify stick figures and the system can rendercharacter Illustrations from the stick figures using a stackedGenerative Adversarial Network (GAN) detailed in FIG. 4H, where twopix2pix-based blocks are stacked to form a stack GAN to generate images.The GAN takes a line drawing and creates an illustration of a person ina pose that matches the line drawing.

The system of FIG. 1C-1F supports computer-aided outlining and firstdraft generation of content. Advantages of starting with the outline inthe system of FIG. 1C may include speed and structure. With a bookoutline aided by AI text suggestions, the writer knows exactly what towrite about next when aided by the AI text agent and the outline canhelp create a solid structure for the novel. The computer canautomatically expand and machine generate concepts for the writer toadopt/edit instantly to overcome blank page syndrome for fictionwriters. Technical writers face similar blocks. Engineers often believetechnical writing to be highly detailed documentation for fellowengineers. While engineers have a great wealth of technical knowledge,they can struggle with how to prepare technical documents, andcommunicating technical information can be just as important asdeveloping it. Further, the writing of computer code can be viewed as atype of technical writing, and businesses have emphasized the need towrite well-documented code. Top-down design means designing from theclient application programmer interface (API) down to the code. The APIlays out a precise functional specification, which says what the codewill do, not how it will do it. Coding bottom up means coding thelowest-level foundations first, testing them, then continuing to buildup. The process of code-writing is similar to writing text for reading,but simply more structured. The system can also be used for transforminginput text to adopt a general style (for example, transforming the textto include a persuasive tone or style), and/or transforming input textto adopt a personal style (for example, transforming the text toencompass the style of any person, if the style is measurable). Exampleuse cases can additionally include transformations involvingvariable-length and/or variable-linguistic complexity (specified asinput) abstractive summarization, as well as domain-driven texttransformations (for example, transforming a technical article onquantum physics to a generic domain text which can be understood by anon-technical person, or vice-versa). As noted above, an abstractivesummarization can refer, for example, to a summarization of an inputparagraph to multiple sentences, while retaining semantic relatedness.Moreover, an abstractive summarization can be carried out across one ormore domains (for example, from a paragraph about quantum physics toplain language English sentences, or vice versa).

In one embodiment, the AI helps the user expand the outline using achatbot (as detailed below) which conversationally engages the user andask the user to expand on where what who why how questions: What? Whatis the concept, topic, or idea? Where? Where does this concept, topic,or idea, apply? Maybe it's an event, or a context situation. Why? Whydoes this matter? Who? Who is this for, or who is involved? When? Isthere a concept of time involved? How? If applicable, how will thishappen? Working through the outline, asking and answering these basicquestions, the computer can build a story that can stretch theimagination. While the foregoing relates to fiction writing, the systemcan be used for non-fiction writing such as for software coding,technical documentation, SEO for web site content, among others.

FIG. 1C shows an exemplary machine programming CAD system. Machineprogramming is a fusion of machine learning, formal methods, programminglanguages, compilers and computer systems. Machine learning and otherautomatic methods are used to create software capable of creating itsown software and is fundamentally about automating software developmentand maintenance. The instant machine programming enables everyone toexpress their creativity and develop their own software without writinga single line of code. The system has a module to determine codesimilarity—whether two code snippets show similar characteristics or aimto achieve similar goals. The AI neural network can determine when twoblocks of code perform a similar computation, even when those blocks usedifferent data structures and algorithms. The system can be configuredto a specific context, allowing it to capture information that describesthe code at a higher level. The neural network can provide more specificinsight into what the code does rather than how it does it. The systemcan assist with incomplete blocks of code that a developer may becurrently writing as part of code recommendation systems or automatedbug fixing. The neural network systems provide similarity scores toblocks of code based on the functional specifications. In other words,if two blocks of code look different in their structure but perform thesame function, the neural networks would classify them as largelysimilar. The system would recognize the intent behind a algorithm inputby a developer and offer candidate codes that are semantically similarbut with improved performance. The system can also automaticallygenerate documentation for the code on behalf of the developer toimprove code documentation.

Turning to FIG. 1D, a technical writing assistant tool is shown. In FIG.1F, the user uploads a series of technical diagrams and enters a briefdescription of the drawings as well as an initial seed text in thedetailed description on what operational aspects or clarification of thesystem being documented. The seed text can be a small paragraph or canbe a detailed text. This system also includes a title, a background textand an abstract/summary text providing useful information along with theseed text that are fed to the AI text suggestion system to generatetopically relevant text suggestions for the user to edit.

Next, the system of FIG. 1E identifies part numbers in the drawings andextracts corresponding part names associated with the part numbers. Thiscan be done automatically using computer vision, OCR engines, or neuralnetworks trained to recognize numbers, among others. One embodiment usesimage recognition to automatically label the part names for the user.

FIG. 1F shows an exemplary system to sketch a tree outline version ofthe current drawing with reference numerals therein. The part names canbe dragged and dropped into the tree structure to generate graphs withnoun phrases (NP), like “vehicle”, “water hose”, “means formanipulating”, “at least two elements” that can be nested under eachother. A parent NP either contains or conceptually covers its childfeatures. A root feature has no parents as it is the main concept.Relations help to define complex technical relationships, that cannot beexpressed simply by nesting features. Relations are nested in the graphsunder features but cannot be nested with each other. A relation mustrefer to its parent feature. A single-feature relation is for example“water hose for watering garden” (defining the purpose of the featurewater hose). A multi-feature relation is for example “water hose isconnected at first end to a water output and at second end to sprinklermeans”.

In another embodiment for FIG. 1F, to help convert the drawing into textwith relationships, the drawings can be rendered lightly as abackground, and the user can move the part names over the section of thedrawing, and the system can auto-generate connection lines or curvesrepresenting a stick figure or a simple distillation of the drawing witha few lines, curves, and dots to graphically illustrate the relationshipconnecting the noun phrases or features. One embodiment shows the userall the part names entered for all figures, then the user can drag it toposition the elements on the tree. The title is shown on top. When themouse is hovered over a particular part name, the corresponding expandedtext from the detailed description section can be shown with reducedopacity. The system can apply graphs and text around dependency treeswhere all the words are kept but the computer sees them in a semanticorder. In other embodiments, the graphs allow compression of text intotext relationships that represent the technical core to the graph. Ifitems are removed from the graph, the described system becomes moregeneral and if something defined similar system before, it would stillbe relevant. Graphs can also be split. If all the pieces of a design arefound from a document, the document should be identified forprecedential work.

In the systems of FIGS. 1E and 1F, the system involves generating theseries of steps that a machine would have to execute to fulfill a user'sintent; in essence, it is the process of generating computer code oralgorithms. This may require discovering new algorithms that are uniqueand different from prior contributions within the same space. In manyinstances, however, invention will be accomplished by identifying how tocombine and adapt known data structures and algorithmic primitives tosolve a particular problem. The graph can be used as a syntacticrepresentation of each program in the search space. Another approachinvolves symbolic search techniques, where the entire program space isrepresented symbolically, either using a special purpose representation,or, in the case of constraint-based synthesis, by reducing it to a setof constraints whose solution can be mapped to a concrete program, whichcan be solved using a SAT or SMT solver or in some cases a numericaloptimization procedure. Deductive synthesis can be used to generatesolution proposals for the user, where the idea is to start with ahigh-level specification and refine it to a low-level implementation byapplying deductive rules or semantics preserving transformations.

FIG. 2A shows an exemplary method to generate a document by providing adocument structure having one or more seed landmark texts therein, eachlandmark text including a milestone overview text and a plurality ofcomponent texts; from the milestone overview text, generating one ormore computer-generated text suggestions to supplement the milestoneoverview text; combining the milestone overview text with each componenttext and generating one or more computer-generated component textsuggestions; and creating the document by combining the milestoneoverview, the one or more computer-generated text suggestions, and eachcomponent text with corresponding one or more computer-generatedcomponent text suggestions.

In implementations, the document structure can be an outline, and eachlandmark text can be a chapter overview, and wherein the component textscomprise a chapter outline. The document can be a fiction or anon-fiction work. The document can be computer code. The document can bea design specification of a new mechanical system. The documentstructure can have one or more figures, wherein each figure comprises abrief description of the drawing, a figure overview, and wherein thecomponent texts comprise a part list of items in each figure. Whensuggesting, the computer text generation can emphasize the componenttext over the milestone overview text when generating the component textsuggestions. The computer combining can include a title and a backgroundtext with the one or more seed landmark texts and providing the combinedtitle, background, and seed landmark texts to a learning machine tosynthesize computer-generated text. The method includes extracting oneor more references from a figure and annotating the one or morereferences with text; and forming one or more computer-generatedreference text suggestions. The method includes performing grammaranalysis and suggesting grammar correction and editing the document forconciseness. The method includes applying a transformer with an encoderthat reads the text input and a decoder that produces a prediction forthe text. The transformer can be a GPT (Generative Pre-trainedTransformer) or a BERT (Bidirectional Encoder Representations fromTransformers) to generate the text.

The first GPT, released in 2018, contained 117 million parameters, thesebeing the weights of the connections between the network's nodes, and agood proxy for the model's complexity. GPT-2, released in 2019,contained 1.5 billion parameters. GPT-3, by comparison, has 175 billionparameters—more than 100 times more than GPT2 and ten times more thancomparable programs and trained on large corpus from the Internet. Thepresent system supplements GPT-3 training data by feeding thetransformers with technical publications and US, EPO and Chineseintellectual property/patent text, and with source code from Github,among others.

For generating alternatives, the method includes determining when twopieces of text, component, module, code, data structure, or imageperform a similar task and showing the determined text, component,module, code, data structure, or image to a user. For designs, themethod includes breaking-down the milestone overview text into one ormore alternate components with different component text but capable ofperforming the milestone overview text based on teachings from prior artdocuments and showing the one or more alternate components as acomputer-generated design around satisfying the milestone overview text,wherein the learning machine learns from prior art and from publiclyavailable data such as Wikipedia and Github.com

One embodiment captures semantically salient properties of the inputcode. The embodiment captures information that describes the context ofthe code (e.g., it is a function call, it is an operation, etc.). Codesimilarity measurement (such as vector dot product, cosine similarity)is used to determine the similarity score between the input program andany other program that has undergone the same code transformationprocess.

One embodiment uses the neural network to map from a goal or intention(given as a set of examples) to a restricted set of components that ithas learned to recognize as useful when satisfying similar goals orintentions. This allows it to then use a synthesizer to solve thesynthesis problem on this restricted program space. The system canhandle complex conditional distributions, allowing it to automaticallydetermine, for example, how to use complex Java and Android APIs. Thesystem includes reasoning at a high-level of abstraction about how thosebuilding blocks fit together, and only then reasoning at the code levelin a targeted fashion. The neural networks model and learn the mappingfrom input-output examples to attributes with: an encoder—adifferentiable mapping from a set of M input-output examples generatedby a single program to a latent real-valued vector, and a decoder—adifferentiable mapping from the latent vector representing a set of Minput/output examples to predictions of the ground truth program'sattributes. The neural network is used to guide the search for a programconsistent with a set of input-output examples instead of directlypredicting the entire source code. Depth-first search (DFS) can searchover programs with a given maximum length. When the search procedureextends a partial program by a new function, it tries the functions inthe DSL in some order. At this point DFS can opt to consider thefunctions as ordered by their predicted probabilities from the neuralnetwork. Another approach is the “Sort and add” enumeration, whichmaintains a set of active functions and performs DFS with the activefunction set only. Whenever the search fails, the next most probablefunction (or several) are added to the active set and the searchrestarts with this larger active set. The neural network is trained fromlarge-scale data sources, such as code repositories like GitHub, orsynthetic data-sources such as randomly generated programs and datasets.

While automation of code is discussed in the above example, anotherexample can identify mechanical or biological modules useful intranslating a desired goal/intent into a practical implementation. Thisis done by analyzing the patent literature and generate design aroundsfrom prior documented solutions in the patent literature. Thisembodiment captures semantically salient properties of the inputrequirement or specification. The system is also context-aware, as itcan capture information that describes the context of the hardwareinvolved. Code similarity detects the similarity score between the inputand any other implementation that has undergone the same mapping ortransformation process. The resulting machine operation feature vectoris provided to the learning machine. The neural network to map from agoal or intention (given as a set of examples) to a restricted set ofcomponents that it has learned to recognize as useful when satisfyingsimilar goals or intentions. This allows it to then use a synthesizer tosolve the synthesis problem on this restricted program space. The systemincludes reasoning at a high-level of abstraction about how thosebuilding blocks fit together, and only then reasoning at the code levelin a targeted fashion.

FIG. 2B shows an exemplary learning system to generate long textdocuments from a summary or given abstract. The system is trained on acorpus of data that includes an abstract and a detailed description.After training, given a new abstract, the system generates a draft forreview. In one embodiment to generate a summary or abstract, the systemincludes the following:

-   -   A sentence tokenizer splits the text into set of sentences.    -   After tokenization, a representation for sentences is done. The        system uses is the Skip-Thought Encoder where the        representations encode the inherent semantics and meaning of the        corresponding sentence. The Skip-Gram Word2Vec is method for        generating the embeddings for words. A weighted average of the        words in the sentences is used to form the sentence embedding.    -   To put the sequence of words in account, the Skip-Thought        sentence encoder is used with two parts, an encoder and a        decoder. The encoder part is a GRU-RNN which generate a fixed        length vector for each sentence. The decoder part takes the        vector representation as an input and tries to generate two        sentences (the next and the previous to it).    -   The encoder-decoder network training minimizes the sentence        reconstruction loss, so that after training the encoder is able        to produce representation of semantically similar sentence that        are closer to each other.    -   After generating the embedding, the next step is to cluster them        into a pre-defined number of clusters. The number of clusters        represents the desired number of sentences in the summary.

In one embodiment with user supervision, the system includes code to:

-   -   extract noun phrases from an abstract;    -   look up corresponding entries in a database such as Wikipedia to        get descriptive text;    -   look up from technical articles and patent databases for        information on the descriptive text;    -   present text from various sources for the user to select;    -   allow the user to refine the abstract and repeat the above steps        if desired; and    -   autogenerate a long article based on user guidance.

In another embodiment, the system includes code to:

-   -   extract noun phrases from an abstract;    -   look up corresponding entries in a database such as Wikipedia to        get descriptive text;    -   look up from technical articles and patent databases for        information on the descriptive text; and    -   autogenerate the article based on user guidance.

In yet another embodiment, the system includes code to:

-   -   extract noun phrases from an abstract;    -   train a neural network such as an RNN to generate description        for the noun phrases from Wikipedia, technical articles and        patent databases for information on the descriptive text; and    -   autogenerate the article.

Advantages of the system may include one or more of the following. Thesystem reduces the cost of writing documents by serving as writingassistants that fill (or in-between) details based on the abstract. Formore technical descriptions where engineering details are important, thesystem can expand from an abstract to a full description with clarity.In other applications that demand flowery language, the efficiency ofhuman drafters can be improved significantly when a master draftergenerates a summary of the major points in the article, and the computerfills in the missing details, similar to inbetweening of animation. Inthe current system, Inbetweening or tweening is a process in all typesof content expansion, including text and video animation. The videoinbetweening includes generating intermediate frames between two images,called key frames, to smoothly transition the first image into thesecond image, where the inbetweens are intermediate drawings whichcreate the illusion of motion from one key frame to the next key frame,all generated using the image transformers. The transformer/learningmachine receives from the user designations on how objects in an imageand can move and change during the tweening process. To aid thetransformer, the user can manually render or adjust transitional framesby hand or software may be used to automatically render transitionalframes using interpolation of graphic parameters. The instant videoinbetweening applies the learning machines to the inbetweening workflowwhere keyframes are generated by a skilled artisan, and then inbetweenmovements are specified for rendering software. The computerizedrenderer does the clean-up and the necessary in-betweens. The system canadapt the detail resolution or rate to the current scene. Detailedpoints may be written on threes or fours chapters of writeups. Differentscenes components of a story might be animated at different resolutionsor rates to conform to the master drafter's command. The result is asignificant speedup in document generation, while cost is reduced.

In another embodiment, the text inbetweening includes generatingintermediate paragraphs between two points in the text outline to evolvesmoothly between the major points in the text outline, where the textinbetweens are intermediate paragraphs, pages, or even chapters whichfollow the text outline to create a cohesive flow as specified by theoutline, all generated using the image transformers or learning machinesto generate long form text as guided by a structure such as an outline.

In one implementation, text preprocessing is the first step for thegiven dataset to prepare it to be an input to the classification model.Cleaning of the dataset is done first using regular expressions (Regex)to remove punctuation and HTML tags. After that, tokenizer is to be usedfor splitting the text into set of words. Text normalization is doneafter tokenization through stemming, lemmatization, and lower-casing ofthe words to convert different forms of each word into one. Then, stopwords are to be removed since they do not carry meaning by themselves(words such as “the”). Finally, words can be checked for their spellingto prevent the chance of having multiple forms of the same word.

Word vectorization or embedding is done after preprocessing to convertthe words into a language understood by any machine learning model,which are numbers. There are two approaches for the problem of sentimentanalysis, either to use supervised machine learning or unsupervisedlexicon-based approach. In one embodiment, semantic word vector spacescan be used in search query can be used where a vector generated fromco-occurrence statistics of a word and its adjacent words is used toencode the meaning of this word. Although word vector models havesucceeded to perform certain NLP tasks such as sentiment analysis, yetthey neglect the compositionality, and context at which these words havebeen used. Thus, they produce misleading, and erroneous results atsentences where long dependencies exist such as sentences which includenegation words or adverbs with similar meanings. Another drawback, wordvectors obtained via co-occurrence statistics consider two factors:syntactic, and semantic similarity so if a small window of context hasbeen used then words like bad, good have very similar representation.

One of the supervised models is Word2Vec which can be included in themodel to be trained in which the parameters of the embedding can betrained with the labels from the labeled dataset. In other embodiments,models that are trained based on part-of-speech tagging, SentiWordNet,for example, to identify the sense of the word and hence a betterembedding. Other vectorizer such as GloVe is trained based on thecontext or aggregated global word-word co-occurrence statistics of theword in the corpus, so it will map the word into the embedding spacebased on its context. In addition, some traditional methods are stillused today such as term frequency-inverse document frequency (tf-idf).Other embodiments apply n-gram, so instead of just using single wordstokens, it can be pairs of example. For example, instead of converting“didn't like movie” into three words, a 3-gram language model can beused to generate triplets of words. The output of the word embedding isfed to the model of the system which can be implemented by enormousmethods. A Support Vector Machine (SVM) or Logistic Regression toclassify the data within the embedding.

In one case, system descriptions are mainly deduced from completesentences rather than words for linguistic reasons by either using amodel that exhibit a memory or using a vectorizer that consider thecontext of the word.

In another case, Abstractive Summarization is done. There are twoapproaches in Abstractive Summarization. The first one is to useSequence-to-sequence RNNs with attention mechanism, the second one is touse a pointer generator networks which is different from a normalSequence-to-Sequence model in that it can copy words from the sourcetext via pointing, which aids accurate reproduction of information,while retaining the ability to produce novel words through thegenerator. It also keeps track of what is summarized to penalizerepetition. Attention mechanism is inspired in the visual attentionanimals have were they focus on specific parts of their visual inputs tocompute adequate responses. Attention used in Seq2Seq architecturesseeks to give more contextual information to the decoder. At everydecoding step, the decoder is informed how much “attention” it shouldgive to the input word, while the transformer model focuses onattention. Positional embeddings provide positional information in thesequence of each element. And despite removing recurrence it stillprovides an encoder-decoder architecture such as the one seen in Seq2Seqmodels.

In yet another implementation, Extractive Summarization is done.Summarization produces a concise and fluent summary while preserving keyinformation content and overall meaning. Historically, researchesstarted the process of automating the process of summarization byintroducing a method that extracts salient sentences from the text usingfeatures such as the work frequency. One implementation introducesweights to the sentences in the documents, ignoring the very frequencycommon words, the same approach that became very basic in most of NLPapplications. The approaches to text summarization vary dramaticallyaccording to the output (extractive or abstractive), purpose (generic,specific domain, or query-based), or the number of documents (single ormany). By generic, we mean that the model makes no assumptions about thedomain or content to be summarized whereas domain-specific assume thatthe text belongs to a specific domain such as biomedical documents. Onthe other hand, query-based summarization produces a summary thatcontains information which answers the input question.

In one embodiment, extractive summarization process identifies the mostimportant parts in the text and produces a subset from the original textthat contain only these parts. However, extractive summarizationreproduces important parts in a new way after interpreting the meaningto generate a new shorter text that convey the critical information fromthe original. Each extractive method should be composed of three mainsteps, construction of an intermediate representation of the input text,scoring the sentences, and selecting a summary comprising of a number ofsentences. There are variations in each approach to the task. Firstly,the common is based on topic words where log-likelihood ratio test toidentify words known as the topic signature. Secondly, afrequency-driven approach can be used as an indicator of importanceusing word probability and Term Frequency Inverse Document Frequency(TF-IDF). Sentences with highest words probability are assumed torepresent the topic of the document and are included in the summary.Using TF-IDF method, the sentences are represented in a space wheredocuments describing same topic are clustered together. Clustercentroids identify the sentences that are central to the topic Thirdly,Latent semantic analysis can be used for extraction. It is anunsupervised method that is similar to the second method but with minormodification on the produced space of clusters. Fourthly, adiscourse-based method can be used to find the semantic relation betweensentences using Cross-Document Structure Theory. However, this relationshould be explicitly made by humans. Fifthly, Bayesian probabilistic isused to infer the words related to a certain topic based on a corpus ofdocuments. Finally, a machine learning approach can be used where wetreat summarization problem as a classification one. In addition, manymodels such as Hidden Markov Models often outperform classificationmethods.

In one implementation, the process is as follows: First, a sentencetokenizer splits the text into set of sentences; and After tokenization,a representation for sentences is done. The system uses is theSkip-Thought Encoder where the representations encode the inherentsemantics and meaning of the corresponding sentence. The Skip-GramWord2Vec is method for generating the embeddings for words. A weightedaverage of the words in the sentences is used to form the sentenceembedding.

To put the sequence of words in account, the Skip-Thought sentenceencoder is used with two parts, an encoder and a decoder. The encoderpart is a GRU-RNN which generate a fixed length vector for eachsentence. The decoder part takes the vector representation as an inputand tries to generate two sentences (the next and the previous to it).The encoder-decoder network training minimizes the sentencereconstruction loss, so that after training the encoder can producerepresentation of semantically similar sentence that are closer to eachother. After generating the embedding, the next step is to cluster theminto a pre-defined number of clusters. The number of clusters representsthe desired number of sentences in the summary.

FIG. 2C shows an exemplary process to create a document that can be afiction or non-fiction work, for example. The process includes:

-   -   Provide a document structure having one or more chapters and for        each chapter add a seed brief descriptive text for the chapter        and add a plurality of sub-plot texts for the chapter    -   For each chapter:        -   From the seed brief descriptive text, generating one or more            computer-generated text suggestions to supplement the seed            brief descriptive text to form a second brief descriptive            text        -   For each sub-plot text, generate one or more            computer-generated component text suggestions based on the            second brief descriptive text and each component text; and    -   Create the document by combining each second brief descriptive        text with each component text with corresponding one or more        computer-generated component text suggestions

FIG. 2D shows an exemplary process to create a storyboard document suchas a movie or animation storyboard, for example. The process includes:

-   -   Provide a storyboard structure having one or more scenes or        pictures therein and for each picture add a seed brief        descriptive text for the picture and add a plurality of sub-plot        texts for the picture    -   For each picture:        -   From the seed brief descriptive text, generating one or more            computer-generated text and computer-generated image            suggestions to supplement the seed brief descriptive text to            form a second brief descriptive text        -   For each sub-plot text, generate one or more            computer-generated component text suggestions and            corresponding pictures based on the second brief descriptive            text and each component text; and    -   Create the storyboard by combining each second brief descriptive        text with each component text with corresponding one or more        computer-generated component text and computer-generated image        suggestions

The structure can be a shot list and the method can take a scene fromthe script and make a shot list, and the system can suggest howparticular camera angles tell the story or make a moment more impactful.The system can suggest revealing details about the characters and thestory via camera angles. The system can autogenerate additional roughsketches of the shot list. The method can help the user to choose one ofthe more complex sequences and scope out a vision for the scene wherethe shots are sketched in the squares of the storyboard, like a comicstrip. One embodiment provides basic shapes and stick figures on a pieceof paper. The system can autogenerate images for the user based onsimilar image search. The system helps the user fill in details. Thestoryboard has the most important elements of each scene. From staticimages, the result is a moving video or animation that follows thestoryline guided by indicated motions or props in the storyboard andcamera angles and framing of each shot. The thumbnails provide a generaloutline of the relevant details of each shot, without going too deepinto distracting details. Once the system receives the images, itauto-suggests additional words at the bottom of the images to give morecontext such as any voice over to add, for example.

FIG. 2E shows an exemplary process to create a technical document suchas an engineering or detailed specification for software coding, forexample. The method includes:

-   -   Provide a document structure having one or more pictures and for        each picture add a seed brief descriptive text for the picture        and add a plurality of component texts like a part-list for the        picture;    -   For each picture:        -   From the seed brief descriptive text, generating one or more            computer-generated text suggestions to supplement the seed            brief descriptive text to form a second brief descriptive            text;        -   For each component text, generate one or more            computer-generated component text suggestions based on the            second brief descriptive text and each component text; and    -   Create the document by combining each second brief descriptive        text with each component text with corresponding one or more        computer-generated component text suggestions

In yet another embodiment for automated computer code generation, thecode inbetweening includes receiving a high level description of apredetermined code (such as pseudo-code), and based on each functionspecified in each line of the pseudocode, generating intermediate codeto perform each sub-function by looking up learned code to achieve thedesired sub-function. If needed, if the pseudo-code line requiresadditional break-down into sub lines to achieve the desiredfunctionality, the transformer can perform the in-line substitution tobreak the desired functionality into digestible sub-tasks to beconverted into computer code that in totality achieve the desired effectfor the code. In this manner, the computer readable code inbetweens areintermediate lines, function calls, module calls, or even entireexternal programs which follow the high level pseudo-code to create acohesive program as specified by the user, all generated using the imagetransformers or learning machines to generate computer code inaccordance with the pseudo-code.

The code can be computer readable code, html code, or hardware ASIC codesuch as ADL or RTL, among others. High-level synthesis tool flows arecan be used for specifying the complete SoC or its constituents.Automatic generation of optimized RTL can be done based on inputspecification and user-directed constraints. The system can start fromopen source processors (RISC V), Coarse-Grained ReconfigurableArchitectures (CGRAs) and Application-Specific Integrated Circuits(ASICs). The transformer is used to generate a high-level synthesis ofASICs based on Architecture Description Languages (ADLs) and theautomated hardware synthesis generated by the transformers/learningmachine can be used to explore intermediate design points between anASIC and a weakly programmable processor, for example.

FIG. 2F shows another exemplary process to create a technical documentsuch as an engineering or detailed specification for software coding,for example. The method includes:

-   -   Provide a document structure having one or more pictures and for        each picture add a seed brief descriptive text for the picture        and add a plurality of component texts like a part-list for the        picture;    -   For each picture:        -   From the seed brief descriptive text, generating one or more            computer-generated text suggestions to supplement the seed            brief descriptive text to form a second brief descriptive            text;        -   For each component text, generate one or more            computer-generated component text suggestions based on the            second brief descriptive text and each component text; and        -   Create the document by combining each second brief            descriptive text with each component text with corresponding            one or more computer-generated component text suggestions

FIG. 2G shows an exemplary process to generate targetedresponses/proposals for the user. The process includes:

-   -   Select deep neural network architecture (for example, retrieval,        generative, and retrieve/refine, transformer-based, BERT-based,        GPT-based, among others) for a learning machine    -   Train the learning machine with data that is logically grouped        or clustered to provide context and accuracy (for example, by        technology field or by industry/specialization; by computer code        such as ASIC code, database code, html code, neural network        code; by type of writing, by type of novel or movie them such as        Mysteries, Romance, Thriller, Science Fiction, Fantasy,        Historical Fiction, among others)    -   Gather customization information from user by interacting with        the user and request DNN to generate context sensitive text        suggestion    -   Determine the trained group or cluster best matching the        customization information and apply the customization        information to bias the learning machine to generate        context-sensitive responses that are realistic in terms of        accuracy and depth

FIG. 2H shows one implementation for generating technology or fieldspecific long form text. The process is as follows:

-   -   First, the field needs to be identified. For example, the        transformer training can be tailored to specific classifications        such as the IPC code. In one embodiment, the process identifies        the international patent classification (IPC) code using various        ways: 1) ask user to indicate or select IPC with graphical user        interface, or 2) auto detect IPC from contextual data    -   Token Bias Process    -   Train BERT (or similar transformer) to classify sections of        patent text (title, summary, abstract, technology field, . . . )        to predict an IPC    -   Given user contextual data (title, summary, abstract, technology        field, . . . ), predict likely IPC    -   Assign token values to each IPC class (outside of vocabulary)    -   Tokenize input text    -   Prepend IPC token to input block—This has the effect of        notifying the model at train and generation time of the IPC        class at each forward pass, thus biasing predictions towards        IPC-specific outputs

One exemplary model has the following parameters: block_size=200,vocab_size=52000, and the ipc parameters are: n_ipcs=1500, the finaldimensions of the model are: block_size=201 and vocab_size=53500

FIG. 2I shows in more details one implementation of the token biasprocess. The token bias includes:

-   -   Collate all contextual data (title, claims, abstract, field of        invention, . . . )    -   Tokenize context via model tokenizer    -   Determine token frequencies as map {token: freq}    -   At generation step, augment token sampling probabilities using a        predetermined policy, for example:        -   P=sampling probability distribution over all tokens in            vocabulary        -   i=initial token prob given prompt        -   f=frequency of token in context        -   d=high-frequency damper (0-1 for damping effect, >1 for            emphasis)        -   a=augmentation constant (user selected)        -   overwrite i:            i<=(i*(f{circumflex over ( )}d))*a            -   (This has the effect of selectively biasing generation                of more frequent tokens) Re-normalize P s.t. sum(P)==1                (required for sampling):                P_aug<=softmax(P)    -   Sample next token using P_aug instead of P.

The foregoing customization of response can be used in otherapplications such as chatbots and SEO optimization.

To optimize for long text generation, the system performs training onthe corpus with a vocabulary of around 52000 words. It then gets asubset of documents (either from a search on terms that is close to thetarget text or from prior history of text generated by the user, forexample) and tokens from the subset of document are then used to biasthe predicted probability to generate the final text. This can be doneby obtaining a histgorgram of tokens, then normalization is done, andthe probability of the subset is merged with the pretrained probability.This increases the probability that new tokens are drawn from the biasedset to increase the likelihood that the neural network generates textmore like a desired text target.

In one embodiment for generating long form text, supplemental text isused to bias the text generator. The text documentation is exported as500 newline-delimited json files. This dataset is far too big to fit inmemory, so a custom encoding script is used to pre-tokenize and storethe dataset in an archive, at a block size of 800 (tokens). The systemused a Transformers Dataset class is used to read this dataset into aneural network model such as the GPT training pipeline. This script ismodified to export a complete model to disk every 500 iterations, sothat model performance can be benchmarked as it trains. Then, using thepipeline API provided in HuggingFace transformers v. 3+, the GPTtokenizer and the custom model is combined into a text generationpipeline using a modified version of the generation_utils.py file, toallow for document biasing as detailed above. This pipeline can performgeneration on a GPU, speeding up generation by 10×.

In another embodiment, the method includes generating long formcontext-sensitive text with a desired token length and targeted at atopic by:

-   -   training a learning machine architecture (LMA) a corpus on a        specific domain (such as engineering, medical, chemical,        patent), wherein the architecture can be BERT, GPT, or a        suitable network, wherein the LMA is trained at the desired        token length (such as 200, 500, 800, or longer token frames of        data) to avoid generating incoherent text whose length is        greater than the desired token length;    -   using a first text input (such as a background or summary or tag        annotations) to retrieve a first set of documents matching the        first text input;    -   applying the first set of documents and the topic as input to        the LMA to generate the context sensitive text with the desired        token length.

For example, a database of patents can be searched to locate documentsmatching the text input, and then the matching documents (or portions ofthe matching documents) can be provided to the LMA to bias the LMA togenerate documents related to the topic. The long text generation can beused as suggested text to the system described in U.S. Pat. No.9,990,351 to the instant inventor, the content of which is incorporatedby reference. The long form text includes suggested text for thebackground, description of figures, description and summary text in thedocument generated by U.S. Pat. No. 9,990,351, for example.

Another embodiment generates context-sensitive text by:

-   -   using a first learning machine to map text matching each topic        to a corresponding vector;    -   building a search index for the search topics and in response to        a search topic returning a responsive first vector;    -   at run time, using a second learning machine to map a topic to a        second vector;    -   determining similarity between the responsive first vector and        the second vector, selecting the most responsive first vector        and retrieving text for the most responsive first vector.

Yet another embodiment generates context-sensitive text by:

-   -   using a first learning machine to map text matching each topic        to a corresponding vector;    -   building a search index for the search topics and in response to        a search topic returning a responsive first vector;    -   training a second learning machine to generate text from a        training corpus;    -   at run time, using the first learning machine to look up the        search index and select responsive documents to provide to the        second learning machine generate responsive context-sensitive        text. The second learning machine can be a learning machine        architecture (LMA) trained a corpus on a specific domain (such        as engineering, medical, chemical, patent), wherein the        architecture can be GPT, or a suitable network, wherein the LMA        is trained at the desired token length (such as 200, 500, 800,        or longer token frames of data) to avoid generating incoherent        text whose length is greater than the desired token length;    -   using a first text input (such as a background or summary or tag        annotations) to retrieve a first set of documents matching the        first text input; and    -   applying the first set of documents and the topic as input to        the LMA to generate the context sensitive text with the desired        token length.

One embodiment blends text from different fields to arrive at acompletely new concept (ideation process). The embodiment uses aTransformer autoencoder, and allows users control over both the globaland local structure of a generated concept sample. In particular, themodel enables using an existing concept or abstract as input to generatea new concept in a similar style, or harmonize a specific new concept ina different technology, but in the style of the original concept. Inother words, given two concepts 1 and 2 each from different patent artunit 1 and 2, the system generates a new concept that is a blend of theconcepts 1 and 2 using a Concept Transformer. The Transformerautoencoder is built on top of the Concept Transformer's architecture asits foundation. As a refresher, Concept Transformer uses relativeattention to better capture the complex structure and periodicitypresent in concepts.

The program encodes abstracts/summaries into idea representations. TheTransformer autoencoder's performance encoder takes as input theabstracts and performs a mean-aggregate of the output embedding to learna global representation of the core concepts. The decoder is allowed toattend to this concept vector.

To harmonize with another input concept, a concept encoder is used inaddition to the performance encoder to embed the respective inputs.These two intermediate representations of melody and performance arethen aggregated to form a single vector input into the decoder.

Instead of using self-attention to operate over absolute positionalencodings of each token in a given sequence done in one embodiment, thepreferred embodiment's Transformer replaces this mechanism with relativeattention and allows the model to keep better track of regularity basedon event orderings.

The standard encoder and decoder stacks of the Transformer have 6 layerswhich are each comprised of a: (1) multi-head relative attentionmechanism; and a (2) position-wise fully connected feed-forward network.The concept encoder takes as input the event-based performance encodingof an input performance, while the melody encoder learns an encoding ofthe melody which has been extracted from the input performance.Depending on the generation task, the encoder output(s) are fed into theTransformer decoder. The decoder shares the same structure as theencoder network, but with an additional multihead attention layer overthe encoder outputs.

At train time the encoder and the decoder use the same inputs, andright-shifting the decoder inputs by one, and doing a single forwardpass through the decoder. At generation time, encoder inputs would bethe original sequence (same at train), and decoder inputs would be thetoken and then loop the decoder to generate a new sequence. One approachis to mask out (set to zero) all encoder outputs which correspond to padtokens, and then (rather than averaging) to stack the tensor along theseq-aka-time (hereinafter ‘time’) axis (where the encoder output is ofshape (batch, time, d_model), and project the resulting (batch,time*d_model) tensor through a feed forward network onto a (batch,d_encoding) space. This is our autoencoded vector, which is used forlater sampling. That vector is then mapped back to the original encodershape via another feed-forward net+reshape (in pytorch, tensor.view( ))step, and the result is used for decoder attentions. In anotherembodiment, during training, the decoder is provided with: (1) theperformance and/or new technology vector representation, which wasmean-aggregated across time, and (2) a perturbed performance sequence.you can think of (2) as the input with some added noise (for NLP tasksthey could look like masking tokens or random word substitutions). The“noisy training” helped quite a bit on this front. The system usesexpanded dataset plus the masking (if present). For the perturbations,masks and substitutions are used.

The DNN generates samples that technologically is similar to aconditioning input performance. The mean-aggregate of the conceptembedding to learn a global representation of concepts. Thismean-performance embedding is then fed into the autoregressive decoder,where the decoder attends to this global representation in order topredict the appropriate new concept. In this way, the generated conceptsare conceptually related yet different due to its application to anotherinventive space or art unit from to the input sequence. The systemapplies two distinct Transformer encoders (each with the samearchitecture) to separately encode the conceptual inputs. The conceptualembeddings are combined to use as input to the decoder.

In combining the intermediate representations, the system can add theconcept embeddings together (sum); or alternatively the system canconcatenate the two embeddings separated with a stop token(concatenate); or alternatively tile the performance embedding acrossevery dimension of technology in the conceptual encoding (tile). Allthree cases works with the mean-aggregated representation of the inputperformance.

To encourage the encoded performance representations to generalizeacross various technology space in different art units, a denoisingautoencoder regularizes the model. For every target concept to betrained, the model is provided with a perturbed version of the inputconcept as the conditioning signal. Finally, the model is trainedend-to-end with maximum likelihood: for a given sequence x of length n,we maximize log pθ(x)=Pn i=1 log pθ(xijx<i) with respect to the modelparameters θ. Training is conducted in an autoencoder-like fashion. Forconceptual conditioning, the Transformer autoencoder is trained topredict a new performance using the combined technology embedding fromtwo or more art units, where the loss is computed with respect to theinput performance.

In one exemplary operation, the user desires to mash up two differentconcepts, and the text command can be: 6G and blockchain and AI. Thesystem responds by inferencing 6G concepts with blockchain concepts andAI concepts to arrive at a new blended concept of using 6G transceiversthat are self-aware and communicate its frequency requirements to nearbytransceiver with the duration, RFpower, and RF frequency. AI is used tooptimize the needs of different transceivers such as proximitycontactless transceivers (RFID), PAN transceivers (Bluetooth), LANtransceivers (WiFi), cellular transceivers (5G/6G), and LEO satellitetransceivers, among others. The transceivers use a mesh network topologyand AI to arrive at an agreed upon transmission schedule which is thenembedded in a blockchain. The system is used to generate and documentIP, such as those disclosed in U.S. application Ser. No. ______ andentitled Smart Wireless Systems by the same inventor, the content ofwhich is incorporated by reference.

One the claim formats are done, the system renders images includingelements recited in each claim. For software claims, the system providesflowcharts that mention all steps. For drawings that require more thanflowchart boxes, the system applies a machine renderer. In alternativeembodiment, a transformer language model receives both the claim textand the inventive drawing input as a single stream of data containing upto 1280 tokens, and is trained using maximum likelihood to generate allof the tokens. In this embodiment, a token is any symbol from a discretevocabulary; for humans, each English letter is a token from a 26-letteralphabet. The system's vocabulary has tokens for illustrated concepts.In one embodiment, each idea abstract/summary is represented using bytepair encoding (BPE) or diagram coding-encoded tokens with a vocabularysize such as 16384. Training is done using the relaxation obviates theneed for an explicit codebook, EMA loss, or dead code revival, and canscale up to large vocabulary sizes. The training can not only generate anew drawing from scratch, but also to regenerate any figure variations,in a way that is consistent with the text prompt such as those by ahuman inventor or by machine.

One embodiment uses a simple decoder-only transformer that receives boththe text prompt and the drawings as a single stream of 1280 tokens-256for the text and 1024 for the concept—and models all of themautoregressively. The attention mask at each of its 64 self-attentionlayers allows each concept token to attend to all text tokens. Astandard causal mask is used for the text tokens, and sparse attentionfor the image tokens with either a row, column, or convolutionalattention pattern, depending on the layer.

In an alternative embodiment, a transformer language model receives boththe text and the inventive concept as a single stream of data containingup to 1280 tokens, and is trained using maximum likelihood to generateall of the tokens. In this embodiment, a token is any symbol from adiscrete vocabulary; for humans, each English letter is a token from a26-letter alphabet. The system's vocabulary has tokens for both text anddrawing concepts learned from patent illustrations. In one embodiment,each concept/drawing is represented using a BPE-encoded tokens with avocabulary size of 16384, and the image is represented using 1024 tokenswith a vocabulary size of 8192. The images are preprocessed to 256×256resolution during training. Similar to VQVAE,1415 each image iscompressed to a 32×32 grid of discrete latent codes using a discreteVAE1011 that we pretrained using a continuous relaxation. The trainingprocedure not only generates an image from scratch, but also toregenerates any rectangular region of an existing image. Thedecoder-only transformer that receives both the text and the image as asingle stream of 1280 tokens-256 for the text and 1024 for the image—andmodels all of them autoregressively. The attention mask at each of its64 self-attention layers allows each image token to attend to all texttokens. Causal mask is used for the text tokens, and sparse attentionfor the image tokens with either a row, column, or convolutionalattention pattern, depending on the layer. In another embodiment, a GANcan be used that is conditioned on text embeddings. The embeddings areproduced by an encoder pretrained using a contrastive loss.

One embodiment can predict from the text generated, the destination of apotential reviewer. This is done by first learning the assignment ofcases based on the text in a document and its assignment to an art unit.Then during inference, the user's text is processed to predict where thecase is likely to be assigned to.

Classification of Patent Texts (and attribution of text weight) is doneas follows:

A RoBERTa-style, encoder-only transformer with a sequence classificationhead (the latter consisting of a Dense feed forward net, a dropoutlayer, and a Dense feed forward net) was trained to predict thefollowing Technology Center classes, for example:TC_CLASS_MAP={

-   -   0: ‘2800—Semiconductors/Memory, Circuits/Measuring and Testing,        Optics/Photocopying, Printing/Measuring and Testing’,    -   1: ‘1600—Biotechnology and Organic Chemistry’,    -   2: ‘2600—Communications’,    -   3: ‘3700—Mechanical Engineering, Manufacturing, Gaming, and        Medical Devices/Processes’,    -   4: ‘1700—Chemical and Materials Engineering’,    -   5: ‘3600—Transportation, Construction, Electronic Commerce,        Agriculture, National Security and License and Review’,    -   6: ‘2100—Computer Architecture and Software’,    -   7: ‘2400—Networking, Multiplexing, Cable, and Security’

}

The dataset was derived from the google public patent dataset. Patentsmatching the above classes were collated to normalize for class-wisetotal text length; specifically, the total word count of all patents inthe least-represented tech center group from the set was calculated, andall other centers were randomly downsampled so as to have equivalenttotal word counts upoin download.

The resulting corpus was then processed by tokenizing each patent withthe standard RoBERTa tokenizer, and splitting the resulting data intolabeled sequences of 512 tokens each (the maximum input size forRoBERTa) and 4096 sequences each (the maximum input size forLongformer). NOTE: The Longformer dataset and models were ultimately notused, as the smoothgrad algorithm used later in the process could notfit the model onto a single GPU (and classification accuracy was notsubstantially greater than the RoBERTa model). Labels were assignedaccording to the TC_CLASS_MAP (above).

Training was performed in parallel on GPUs, with fp16 mixed precision(AMP) and gradient accumulation, for an effective batch size of 128.Training was stopped when the evaluation set F1 score started todiverge, and the best model (picked for highest eval F1) was selectedfor use to predict art unit assignment.

The SmoothGrad algorithm is utilized at prediction time to return boththe predicted text label, and a token-wise impact attribution on thatprediction. Tokens with higher impact on the prediction are more red.The result is a color coded output that indicates what words are morelikely to impact the assignment of a case to an art unit or technologycenter:

-   -   . . . an iPhone with a geometric shape to find an English word        puzzle. Thus it is clear that it is desirable to provide a        system of word puzzle game application in iPhone in a better        enhanced way. The present invention overcomes these and other        problems by providing software game application in an iPhone.        Further it will be apparent to those skilled in the art that the        objects of this invention have been achieved by providing a        software application game in an iPhone which consists of a        English dictionary words that forms as a puzzle word game with a        geometrical shape which is unique in nature unlike existing        mobile puzzle game that are suited only for limited purposes.        Various changes may be made in and without departing from the        concept of the invention. Further, features of some stages        disclosed in this application may be employed with features of        other stages. Therefore, the scope of the invention is to be        determined by the terminology, and the legal equivalents        thereof. SUMMARY OF THE INVENTION This present invention may be        summarized, at least in part, with reference to its objects. The        foremost objective of this invention is to provide a system of        word puzzle game application in an iPhone. Another objective of        this invention is to entertain an iPhone user by providing a        word puzzle game with geometrical shapes. Another objective of        this invention is to educate the user of an iPhone with a simple        Label: 3700—Mechanical Engineering, Manufacturing, Gaming, and        Medical Devices/Processes |93.89%|

One embodiment provides an interactive tool where the user can globallychange a word and see the impact of the assignment. That way, the usercan influence the art unit assignment as desired.

FIG. 3A shows a chatbot system that applies the above methodology toanswering user questions on an automated basis, thus greatly reducingcost and increasing customer convenience due to its ability to resolveissues 24×7. The process is as follows:

-   -   Select deep neural network architecture (for example, retrieval,        generative, and retrieve/refine, transformer-based, BERT-based,        GPT-based, among others) for a learning machine    -   Collect training data and update on periodic basis:        -   Store non-public information into a database from a site            desiring to have a chatbot to answer questions, including            CRM databases for common user questions and non-public            product maintenance or service information for products        -   Crawl web site of the company desiring to have the chatbot            to answer questions to extract user manuals, FAQs and all            publicly available text        -   Crawl fan sites or product review sites for information            about company/product/service        -   Crawl competitor sites to extract industry text        -   Crawl the internet for any mention of the company name or            product names including negative reviews and flag such            reviews for company responsive text as training data    -   Train learning machine with data that is logically grouped or        clustered to provide context and accuracy (for example, by        technology field; by product; by customer type (engineers,        housewife, student, . . . , or by industry/specialization, etc)        and periodically update training with new data    -   Gather customization information from user by interacting with        the user and retrieving prior interactions with the user and        prior purchases and complaints/returns by the user    -   Determine the trained group or cluster best matching the        customization information and apply the customization        information to bias the learning machine to generate        context-sensitive chats that are optimized to answer or interact        with the user    -   Detect user emotions during the interaction based on user facial        expression (periodic sampling of camera image and/or verbal        expression), or based on text response by user, or by explicit        happiness rating next to the chat box text entry space    -   If user is satisfied with the interaction based on detected        emotion, continue responding/chatting    -   If user is dissatisfied based on detected emotion, select a        call-center agent best matched to the user profile or need and        transfer to selected agent at a call center

One embodiment employs the poly-encoder architecture which encode globalfeatures of the context using multiple representations (n codes, where nis a hyperparameter), which are attended to by each possible candidateresponse. This final attention mechanism gives improved performance overa single global vector representation (so-called “biencoders”), whilststill being tractable to compute compared to simply concatenating inputand output as input to a Transformer (or “crossencoders”). A Seq2SeqTransformer architecture is used to generate responses rather thanretrieve them from a fixed set. One implementation is based on theParlAI version with Byte-Level BPE tokenization trained on thepre-training data, as implemented in HuggingFace's Tokenizers.

To avoid producing dull and repetitive chat responses, given thedialogue history, the retrieval model is first used to produce a draftresponse which is then appended to the input sequence of the generator,along with a special separator token. The generator then outputs aresponse as normal given this modified input sequence. Alternatively,the system can retrieve from a large knowledge base, instead ofretrieving an initial dialogue utterance and then condition thegeneration on the retrieved knowledge. The same retrieval system uses aTF-IDF-based inverted index lookup over the collected/crawled data toproduce an initial set of knowledge candidates. A Transformer retrievermodel is then used to rank the candidates and select a single sentencewhich is used to condition generation. A Transformer-based classifier istrained to choose when to perform retrieval or not on a per-turn basis,as some contexts do not require knowledge. This was trained as atwo-class classifier discriminating between contexts that requireknowledge or not in the fine-tuning tasks.

The domain specific training of the learning machine enables it to havein-depth knowledge if sufficiently interrogated. The system usesindustry specific jargon due to the domain training so that it does notuse generic/simpler language and it does not repeat oftused phrases.

In FIG. 4H, the system uses classifiers of toxic language trained onadversarial toxic data that fools existing classifiers and is then usedas additional data to make them more robust. The classifier at test timeto detect toxic language before it is rendered by the chatbot. Thesystem also mitigates race and gender bias in dialogue throughconditional generation, controlling the amount of racial or genderedwords to be more neutral.

In the user emotion detection, the chatbot can request access to cameraand microphone (mike). If permitted, a variety of analysis can be done,but if not, text-based emotion analysis can be done. The system usesdeep learning to recognize emotional intent patterns in human text,speech and facial expressions and respond to those cues in appropriate,empathetic ways—such as offering directions or information. Sentimentanalysis for understanding the underlying feelings and emotions inopinions, whether written or spoken. One embodiment uses thetransformers described herein and trained to analyze emotion based onthe video/sound/text. A transformer model is used to fuseaudio-visual-text modalities on the model level. A multi-head attentionproduces multimodal emotional intermediate representations from commonsemantic feature space after encoding text, audio and visual modalities,as supplemented by long-term temporal dependencies with self-attention.

If camera/mike access is allowed, facial analysis for frowning and voicepitch analysis and text sentiment analysis can be done in oneembodiment. In other embodiments, posture, what's happening in theenvironment, physiological information such as what's going on with thenervous system, and smile context detection on a specific person in aspecific situation can be done. Additionally, patterns in people withsimilar characteristics like gender sampled across cultures can be doneto increase emotion detection accuracy. A number of emotional detectionmodules can be used, for example: DELTA is a deep learning based naturallanguage and speech processing platform; Emotion Recognition NeuralNetworks using DNN with tensorflow; Emopy—deep neural net toolkit foremotion analysis via Facial Expression Recognition (FER); EmotionRecognition—Real time emotion recognition; Speech Emotion Analyzer—Theneural network model is capable of detecting five different male/femaleemotions from audio speeches. (Deep Learning, NLP, Python); ConyEmotion—This repo contains implementation of different architectures foremotion recognition in conversations; Deepface—A Lightweight Deep FaceRecognition and Facial Attribute Analysis (Age, Gender, Emotion andRace) Framework for Python; Emotion Detection—Real—time Facial EmotionDetection using deep learning; Emotion—Recognizes human faces and theircorresponding emotions from a video or webcam feed; and MultimodalEmotion Recognition—A real time Multimodal Emotion Recognition web appfor text, sound and video inputs; among others, the content of thedocumentations from their respective github sites areincorporated—by-reference.

For text only analysis, one embodiment uses the vaderSentiment packagethat provides a measure of positive, negative, and neutral sentiment.For given input text data, vaderSentiment returns a 3-tuple of polarityscore percentages and a single scoring measure, referred to asvaderSentiment's compound metric. Other suitable sentiment analysistools can be used.

If user dissatisfaction is detected, the system forwards the user to acall center agent using a selection process determined by the learningmachine trained for routing users to agents includes rating agents onperformance or success of agent data and caller data, or both. Thechecking for optimal interaction includes combining agent workperformance, agent demographic/psychographic data, and other workperformance data (“agent data”), along with demographic, psychographic,and other business-relevant data about callers (“caller data”). Agentand caller demographic data can be: gender, race, age, education,accent, income, nationality, ethnicity, area code, zip code, maritalstatus, job status, credit score, for example. Agent and callerpsychographic data can cover introversion, sociability, work/employmentstatus, film and television preferences, among others.

FIG. 3B shows a chatbot system that applies the above methodology torouting a caller to a predetermined call center agent to optimizeconversion, sales, or any other business goals. The process is asfollows:

-   -   Select deep neural network architecture (for example, retrieval,        generative, and retrieve/refine, transformer-based, BERT-based,        GPT-based, among others) for a learning machine    -   Collect training data and update on periodic basis:        -   Store non-public information into a database from a site            desiring to have a chatbot to answer questions, including            CRM databases for common user questions and non-public            product maintenance or service information for products, and            CRM databases for customer profiles and agent profiles        -   Crawl web site of the company desiring to have the chatbot            to answer questions to extract user manuals, FAQs and all            publicly available text        -   Crawl fan sites or product review sites for information            about company/product/service        -   Crawl competitor sites to extract industry text        -   Crawl the internet for any mention of the company name or            product names including negative reviews and flag such            reviews for company responsive text as training data        -   Train learning machine with data that is logically grouped            or clustered to provide context and accuracy (for example,            by customer profile; by agent grade/performance, by            agent-caller interaction history; by technology field; by            product; by customer type (engineers, housewife, student, .            . . , or by industry/specialization, etc) and periodically            update training with new data        -   Gather customization information from user by interacting            with the user and retrieving prior interactions with the            user and prior purchases and complaints/returns by the user    -   Determine the trained group or cluster best matching the        customization information and apply the customization        information to bias the learning machine to generate        context-sensitive chats that are optimized to answer or interact        with the user    -   Route caller to select agent based on trained learning machine    -   Detect user emotions during the interaction based on user facial        expression (periodic sampling of camera image and/or verbal        expression), or based on text response by user, or by explicit        happiness rating next to the chat box text entry space    -   If user is satisfied with the interaction based on detected        emotion, continue agent-caller interaction    -   If user is dissatisfied based on detected emotion, select        another call-center agent best matched to the user profile or        need and transfer to new selected agent or supervisor        (escalation of service)

The training data includes caller data associated with one or morecallers (e.g., a caller on hold), agent data associated with one or moreagents (e.g., one or more available agents). Caller data (such as acaller demographic or psychographic data) is determined or identifiedfor a caller. The system can get caller data from available databases byusing the caller's contact information as an index. Available databasesinclude, but are not limited to, those that are publicly available,those that are commercially available, or those created by a contactcenter or a contact center client. If the caller's contact informationis not already known, caller data can be retrieved from the CallerIDinformation or by requesting this information of the caller at theoutset of the contact, such as through entry of a caller account numberor other caller-identifying information. Other business-relevant datasuch as historic purchase behavior, current level of satisfaction as acustomer, or volunteered level of interest in a product may also beretrieved from available databases. Agent data includes agent grades(which may be determined from grading or ranking agents on desiredoutcomes), agent demographic data, agent psychographic data, and otherbusiness-relevant data about the agent (individually or collectivelyreferred to in this application as “agent data”), along withdemographic, psychographic, and other business-relevant data aboutcallers (individually or collectively referred to in this application as“caller data”). Agent and caller demographic data can comprise any of:gender, race, age, education, accent, income, nationality, ethnicity,area code, zip code, marital status, job status, credit score, and thelike. Agent and caller psychographic data can comprise any ofintroversion, sociability, desire for financial success, film andtelevision preferences, and the like. One method of determining agentdemographic or psychographic data can involve surveying agents at thetime of their employment or periodically throughout their employmentsuch as agent grades, demographic, psychographic, and otherbusiness-relevant data, along with caller demographic, psychographic,and other business-relevant data. The learning machine matches eachcaller with each agent and estimates the probable outcome of eachmatching along a number of optimal interactions, such as the generationof a sale, the duration of contact, or the likelihood of generating aninteraction that a customer finds satisfying.

The exemplary method may include determining caller data associated withone or more callers (e.g., a caller on hold), determining agent dataassociated with one or more agents (e.g., one or more available agents),comparing the agent data and the caller data with the transformers, andmatching the caller to an agent to increase the chance of an optimalinteraction. The learning machine predicts and recommends optimalinteractions for every agent against every available caller.Alternatively, the computer model can comprise subsets of these, or setscontaining the aforementioned sets. For example, instead of matchingevery agent logged into the contact center with every available caller,examples can match every available agent with every available caller, oreven a narrower subset of agents or callers. Likewise, the presentinvention can match every agent that ever worked on a campaign—whetheravailable or logged in or not—with every available caller. Similarly,the computer model can comprise predicted chances for one optimalinteraction(s).

If best match is no possible, conventional routing via an Automatic CallDistribution (ACD) queue order or the like is done by determining aqueue order of the caller. For example, if other callers are on holdwaiting for an available agent, the caller may be queued with othercallers, e.g., a system may order the callers in terms of hold time andpreferentially map those callers that have been holding the longest. Thesystem then maps the agent that has been waiting or idle the longestwith the caller that has been holding the longest. The caller may thenbe routed to the agent. The system can preferentially route callers tothose agents shown to have greater ability to generate sales, canincrease the chances of achieving greater sales during the contacts.Similarly, other agents may be shown to generate shorter interactionswith callers than that of other agents at the same contact center. Bypreferentially routing contacts to the agents shown to generate shorterinteractions with callers, a contact center or contact center client candecrease its overall need for agents and communication bandwidth, andtherefore, reduce its costs.

FIG. 3C shows an exemplary search engine optimization (SEO) system. Theprocess is as follows:

-   -   Select deep neural network architecture (for example, retrieval,        generative, and retrieve/refine, transformer-based, BERT-based,        GPT-based, among others) for a learning machine    -   Collect training data:        -   Gather customization information from user by collecting web            site map and proposed web site content for new web site            design, or by crawling an existing web site, focusing on            frequently asked questions (FAQs) and question and answers            (Q&As) and all publicly available text        -   Gather marketing input including marketing positioning, top            keywords/semantic concept/questions to be ranked from SEO            tools identifying top keywords are being used and what            questions are being asked to create high quality content            (system can handle target keywords with accurate keyword            volume and difficulty metrics)        -   Crawl competitor sites to extract industry text        -   Crawl the internet for any mention of the company name or            product names    -   Train the learning machine with data that is logically grouped        or clustered to provide context and accuracy (for example, by        technology field; by product; by customer type (engineers,        housewife, student, . . . , or by industry/specialization, etc)    -   Determine the trained group or cluster best matching the        customization information and apply the customization        information to bias the learning machine to generate        context-sensitive structured data markup.    -   Generate proposed web content that anticipates answers and        solutions in the content and grow the authority of the domain    -   Generate Semantic Knowledge Mapping and schema markup for        crawlers to use    -   Test SEO performance, and generate new text and repeat until SEO        performance reaches a predetermined target

The content generator suggests contents for the Website that are TopicRelevant, enabling website to be relevant to the topic and everythingthat is related and useful. High-scoring web pages do more than justprovide sales copy or direct answers to questions. They also containsupporting information. Many times, one answer surfaces another questionfrom the reader, so the system provides related answers and anticipatetheir needs. Include information the company knows customers willneed—and haven't thought of before. This can be done with the customtraining data such as frequently asked questions (FAQs) and question andanswers (Q&As) related to the industry overall and specifically thecompany.

The software provides a structured approach to content creation combinedwith structured data markup. The software anticipates answers andsolutions in the content and grow the authority of the domain to grow.In one embodiment, search tools such as Moz are queried on a periodicbasis and the system can update its semantic knowledge map to generatecontent with the following:

1. Research user signals to create a list of questions asked.

2. Narrow the target audience and the top questions asked.

3. Use Jump Links to take viewers immediately to answers

4. Match and organize answers.

5. Optimize existing content for conversational phrases.

6. Provide answers to all top related questions.

7. Add semantic-rich search terms to content.

In addition, the system can convert existing web site content with thefollowing:

-   -   Content Improvement: Rewriting web content with more        conversational language.    -   Featured Snippets: Optimizing on page content to earn featured        snippets atop organic results.    -   Schema Markup: Using structured data markup to tag elements of        web pages and help search engines more accurately interpret        them.

Production questions such as size, color, what a product is made of,etc., are things people are asking. Consumers are asking more questionsrelated to a specific product before making a purchase. Follow theinstructions carefully when implementing product markup. The systemincorporates JSON-LD markup when possible and fitting. Reviewers oftenanswer the questions other buyers are likely to ask. The systemgenerates wording that aligns which purchase intent. On top of the page,the system creates a table of context, each jump-link taking the user tothe part of the page answering each question. Creating jump links makesthe work easier for a site visitors to quickly see just the answer thatthey want. Jump links to specific answers lessens your chances of a lowbounce rate and improves crawling and indexing. The system automaticallymaintains the accuracy and freshness each product item's schema.Maintaining a correct schema helps site's content get featured in thePAA and for additional Related Questions.

Users want the best matching, concise answer immediately. With so manyquestions being asked, the system deciphers which answers are mostneeded. This helps structure the order for creating or optimizing thatcontent. Voice searches are more conversational by nature when evaluatedto text searches. Local searchers questions most often fall in thissegmentation. When on the go and a need arises, people tend to speak aquery. The system generates semantic knowledge mapping for both mobileand desktop search experience. The content generated by the systemprovides the audience with a road map to help them along their purchasejourney. The common questions asked may vary at each stage; many fitlong-tail keywords. For example early on consumers will likely be pricecomparison shopping, so their questions will center on value and use.Before pulling the trigger on a purchase, they may be asking aboutreturn policies and means of shipping.

The system generates Semantic Knowledge Mapping and generates contextuallanguage instead of verbatim keywords. It focuses on the whole contextof searcher's queries. The content length is controlled to match asearcher's intent which differs for detailed informational content and aquick answer in summary form. The system provides a semantic analysis ofthe natural language content, the system assists the web site contentcreator to locate the words in the creator's original content thatcapture the real meaning of the original text and then suggests textelements to assign to their logical and grammatical role and buildrelationships between different concepts in the text that align withBERT.

The system can apply a knowledge-based library of concepts to helpsearch engines detect different businesses or entities are ‘Known for’or to define entities better connected relationships. Web pages forspecific entities may gain top positioning in search results when userengagement history indicates that search intent may include that entitywithin a query. The Natural Language system discerns syntax, entities,and sentiment in text, and organizes text into a predefined set ofcategories. The resulting content is also highly succinct, with morefactual content that is written by authoritative sources. It is alsoengaging.

The system can transform “traditional SEO copy-writing” to better matchthe SEO's semantic search and update the Knowledge Graphs, entities. Thesystem is optimized for the Searcher who Relies on Voice-ActivatedSearching which changes their search behavior from text input to spokeninput. The system converts the original text into structured data markupthat fits the context with entities along with their unique identifierswhich may be used to help describe the content to search engines.

The system generates snippets, structured data, and knowledge graphs toanswer people's questions and to convert the website's answers intofeatured snippets. Generating fresh and unique answer-rich contentimproves placement as a featured snippet. This is one means of givingthe assistants more answer response material to match to spoken queries.

The text generation generates ontological markups or schema markups forentities on web page content, relationships to other entities, theirconnected relationships to attributes (properties) about those entitiesand the relationships to entity classifications. The systemautomatically generates a site's architecture, ontologies, andstructured data. The system can handle Query Segmentation related tosegmenting out a specific query into units of a smaller size. The systemcan perform custom entity modeling—especially because entityunderstanding helps us communicate better with real consumers. Theentities provide search engines with a better and deeper understandingof topics which in turn, enable information about the Entity to bedelivered in any language (with live translation if necessary), sincelanguage has only a supportive role for the query—like a modifier.Whatever Entity Understanding and Entity Relationships the search enginelearns in one language can automatically be translated to otherlanguages in the Knowledge Graph. The computer-generated markups areoptimized for Direct Answers or direct answers to queries, similar aFeatured Snippet. The system provides correct product/service markup andanchor text to assist gaining the position of answering the query.

The computer-generated text leverages the transformer chat-bot contentsthat are conversational in nature. The content produced for a website orblog incorporates conversational language. With conversational sentencesintegrated into a website's content, it will be simpler for users tofind information on those subjects using text or voice search. Afeatured snippet is a block of text an SEO shows on the top of organicresults for question queries, and the snippet can be used for voiceassistant response.

The FAQs are provided with a question and answer schema to the FAQ asfeatured snippets. Schema code enables search engines to extract factsand information about entities for matching queries better. The site canassociate the relationships between its content entities to theirattributes and classifications. A confidence scores is then generatedform relationships and added to Google's library of answers it may drawfrom. It not only identifies each page's highlights but is aware ofnotes, media elements, reviews and such within them, too.

One embodiment optimizes the SEO content for featured snippets. Searchengines programmatically determines that a page contains a likely answerto the user's question, and displays the result as a featured snippetdisplayed in typical search results and are accentuated with a speciallayout. Begin by determining what is a simple, straightforward questionin your market space. Then, craft an equally simple and straightforwardanswer to that question. The content generated is a full answer to thequestion and address related issues with that particular question andanswer occurring somewhere on that page in a very focused spot in theformat of an itemized list or a paragraph shortening the answer so thatcomputers and viewers can quickly spot it on the page. The domain has astrong trusted authority factor for featured snippets and the KnowledgeGraph.

FIG. 3D shows an exemplary system to respond to infectious outbreaks.One embodiment provides a chatbot to provide advice to patients of aninfectious disease such as COVID19. Such chatbot may get the U.S. Foodand Drug Administration (FDA) 510(k) and European CE approval for publicuse. The system applies a trained chatbot operating in concert withmobile fitness monitoring and contact tracing to assist users inanswering their health questions in an efficient timely manner thatminimizes compute resources and health professional time to free them upfor ICU patients, for example. The system can receive FDA 501k or CEclearance approval. The chatbot crawls official governmentcommunications about COVID-19 from governments and the World HealthOrganization as well as predetermined vetted sources, the chatbot inconjunction with a mobile app assesses known symptoms and answersquestions about government policies.

The process starts with the appropriate deep neural network architecture(for example, retrieval, generative, and retrieve/refine,transformer-based, BERT-based, GPT-based, among others) for a learningmachine, and then performs the following:

-   -   Collect training data and update on periodic basis:        -   Store non-public information into a database from a site            desiring to have a chatbot to answer questions, including            hospital databases for patient private data and databases            containing mobile fitness tracking devices for users        -   Crawl web site of WHO, government agencies, and            predetermined research institutions knowledgeable about            infectious diseases to extract instructions, frequently            asked questions (FAQs) and question and answers (Q&As), and            all publicly available text        -   Crawl the internet for any mention of            solutions/methods/product names including negative reviews            as training data    -   Train learning machine with data that is logically grouped or        clustered to provide context and accuracy (for example, by age,        sex, race, home location, health history, social economics,        risks for lung or breathing diseases etc) and periodically        update training with new data    -   Gather customization information from user by collecting recent        data from mobile fitness devices and by interacting with the        user and retrieving prior interactions with the user and prior        health reports by the user, as well as by the clusters of people        the user is affiliated with    -   Determine the trained group or cluster best matching the health        condition information and apply the customization information to        bias the learning machine to generate context-sensitive chats        that are optimized to answer or interact with the user regarding        symptoms    -   Detect user emotions during the interaction based on user facial        expression (periodic sampling of camera image and/or verbal        expression), or based on text response by user, or by explicit        happiness rating next to the chat box text entry space    -   If user is satisfied with the interaction based on detected        emotion, continue responding/chatting    -   If user appears ill, upset or exhibits unusual behaviors not        observed before, request opportunity to have a health        professional to follow up at later time, or optionally select a        call-center agent best matched to the user profile or need and        transfer to selected agent at a call center for assistance.

As part of the analysis, the chatbot detects users with higher risk suchas users with suppressed immune systems (cancer treatment or who haverecently had an organ transplant), unvaccinated users that may besusceptible against common infectious diseases, healthcare workers,users who are at or traveling to at-risk areas where they may be exposedto mosquitoes that carry pathogens, among others.

In one implementation, vital signs from smart watches can be used tomonitor core body temperature pattern, breathing pattern, coughingpattern, and walking/exercise patterns to detect changes indicative ofan infectious disease. The breathing rate/pattern can be detectedthrough EKG or other means. The coughing pattern can be detected bysound using a microphone, or can be done through body motions asdetected by accelerometers, which also detect the walking/exercisepatterns. Contact tracing can be done to detect group activities andassociated people to see if there are group activities indicative of anoutbreak in the group. Communications with members of such group arealso used to infer on-set of the disease among the group.

Such information can be used when the chatbot asks the user forsymptoms. Symptoms of infectious disease are particular to the type ofdisease. For example, Symptoms may appear 2-14 days after exposure tothe virus. Symptoms of COVID-19 may include Fever or chills, Cough,Shortness of breath or difficulty breathing, Fatigue, Muscle or bodyaches, Headache, New loss of taste or smell, Sore throat, Congestion orrunny nose, Nausea or vomiting, Diarrhea, Trouble breathing, Persistentpain or pressure in the chest, New confusion, Inability to wake or stayawake, or Bluish lips or face, according to the CDC. Symptoms ofinfluenza include: Fever, Chills, Congestion, Fatigue, Muscle aches andheadache. Other infectious diseases, such as Shigella, cause moreserious symptoms, including Bloody diarrhea, Vomiting, Fever,Dehydration (lack of fluid), and Shock.

The system also helps patients with chronic conditions, many of whom areforegoing urgent care out of fear of getting Covid-19 at the hospital.For example, if the user's medical history shows hypertensive from thedata, and if the user is not being treated for or charged for high bloodpressure medicine, the system can alert the doctor and suggest medicinefor their hypertension. The chatbot can detect situations maybe theywere taking it, stopped taking it, and they haven't gotten a refillbecause of Covid.

The chatbot can serve factual answers to user's questions. Users oftenquery a search engine with a specific question in mind and often thesequeries are keywords or sub-sentential fragments. The chatbot may relyon multiple methods to measure the matching degree between a questionand an answer candidate.

The system becomes a source for trusted information on a topic ofinterest to the site clients and prospective buyers means that the webpages are successfully putting the user experience first. Schema, asemantic vocabulary of tags (or microdata), can be added to a site'sHTML code to enhance search engines' ability to read and represent webpages in SERPs. While rich snippets do not directly influence a site'srankings, structured data markup to enable rich snippets may generateindirect SEO paybacks by making your page more effortlessly indexable.It also informs search engines about what's important to you in yourcontent and does a better job with accurate and targeted metadata. Themarkup provides search engines with better structured content which inturn it can use to provide answers to searchers. It can affect rankingsin SERPs and improve the domain authority of the website by indirectlyinfluencing the page's visibility through SERP featured snippets.

ClaimReview Schema markup is used to help search engines interpret yourpages to fit the context of a search query. At a high level,claimReviewed, claimUrl, claimUrilOriginal are all attributes ofClaimReview. The system can use Google Data Search is surfacing newdatasets that can be sourced to back up the computer-generated text'sclaims. Claim Review-based factcheck markup defines a structure thatcorresponds to the kind of information included in many fact-checkingpages. The fundamental notion is a ClaimReview has an author(schema.org/author), which is typically an Organization(schema.org/Organization) (i.e. the fact checking organization orpublisher), but could also be a Person (schema.org/Person). TheclaimReviewed (schema.org/claimReviewed) property of a ClaimReview(schema.org/ClaimReview) summarizes the claim being reviewed. This mayinclude clarifications of the original wording to addressintelligibility, civility, context or brevity, and can includetranslations. This value of the claimReviewed (schema.org/claimReviewed)property is typically a simple textual string (but could be a Claim(schema.org/Claim) with a text (schema.org/text) property, although thisis not encouraged). The itemReviewed (schema.org/itemReviewed) propertyof ClaimReview (schema.org/ClaimReview) indicates specificmanifestations of the claim being reviewed. This can either be a Claim(schema.org/Claim) [preferred] or [historically] a CreativeWork(schema.org/CreativeWork) within which the claim is described orreported. The value of itemReviewed (chema.org/itemReviewed) (preferablya Claim (schema.org/Claim) to avoid ambiguity) has an author(schema.org/author), which is a Person (schema.org/Person) orOrganization (schema.org/Organization) that has made the claim. A Claim(schema.org/Claim) can be associated with a CreativeWork(schema.org/CreativeWork) it occurs in, using the appearance(schema.org/appearance) or firstAppearance(schema.org/firstAppearance)properties. This is preferable to describingappearances using itemReviewed (schema.org/itemReviewed) as itdistinguishes more explicitly between the author (schema.org/author) ofthe Claim (schema.org/Claim) versus author (schema.org/author) ofmaterials discussing those claims. The reviewRating(schema.org/reviewRating) property of the ClaimReview(schema.org/ClaimReview) indicates a Rating (schema.org/Rating) of theclaim. A rating can be summarized textually with a alternateName(schema.org/alternateName) property, and with a numerical rating on ascale from worstValue (schema.org/worstValue) (lowest) to bestValue(schema.org/bestValue) (highest). The author (schema.org/author) (orcreator (schema.org/creator), publisher (schema.org/publisher) of aClaimReview (schema.org/ClaimReview), or of a Claim (schema.org/Claim),or CreativeWork (schema.org/CreativeWork), can be either an Organization(schema.org/Organization) or Person (schema.org/Person).

In another embodiment, the sensor(s) can collect vital signs such astemperature, heart rate, ECG, EEG, PPG, and bioimpedance, among others.For example, in one aspect, a system includes a cellular, WiFi, or andBluetooth or UWB transceiver coupled to a processor; an accelerometer ora motion sensor coupled to the processor; and a sensor coupled to theprocessor to sense mood body vital sign, wherein text, image, sound, orvideo is rendered in response to a sensed mood or body vital sign; and awearable device operating wirelessly with the processor, wherein thewearable device includes at least one sensor coupled to a back of thewearable device and wherein the wearable device recognizes and executesthe speech command. In another aspect, a mobile system, comprising: atransceiver to communicate data via a personal area network (PAN); anaccelerometer and a gyroscope; a processor coupled to the transceiver,the accelerometer and the gyroscope, the processor executing one or moreapplications to record user speech and to record data regarding movementdetected by the accelerometer and the gyroscope; two or more sensors incommunication with the processor to detect user vital sign data; and ahealth application executed by the processor to generate a healthanalysis using the vital sign data and the data regarding movementdetected by the accelerometer and the gyroscope, wherein the transceivercommunicates the analysis to another computer via the PAN.

In yet another aspect, a system includes a processor; a cellular, WiFi,or Bluetooth or UWB transceiver coupled to the processor; anaccelerometer or a motion sensor coupled to the processor; and a sensorcoupled to the processor to sense mood, wherein text, image, sound, orvideo is rendered in response to the sensed mood. In another aspect, asystem includes an accelerometer to detect movement or fitness; a sensorcoupled to a wrist, hand or finger to detect blood-oxygen levels orheart rate or pulse rate and mounted on a wristwatch wearable device anda voice communication device having a wireless transceiver adapted toreceive blood-oxygen level or heart rate or pulse rate from the sensorover a wireless personal area network (PAN). In yet another aspect, asystem includes a cellular telephone having a vital sign sensor thereonto detect heart rate, pulse rate or blood-oxygen levels; and awristwatch wearable device in wireless communication with the cellulartelephone, including: a sensor coupled to a wrist, hand or finger todetect blood-oxygen levels, heart rate or pulse rate; a wirelesstransceiver adapted to communicate with the cellular telephone over awireless personal area network (PAN); and a processor coupled to thesensor and the transceiver to send pulse rate to the cellular telephone.In a further aspect, a health care monitoring system for a personincludes one or more wireless nodes forming a wireless network tocommunicate data over the wireless network to detect a health problem.Implementations can include watches that capture fitness data (activity,heart rate, blood pressure, walking rate, dietary or calorieconsumption, among others) and sending the data to a hospital databasewhere medical and fitness data is used to treat the patient. Otherimplementations include collecting data from different devices withdifferent communication protocols such as blood pressure measurementdevices, scales, glucose meters, among others, and upload the data to acomputer which converts the data into an intermediate format that iscompatible with different protocols for interoperability purposes. Inanother aspect, a heart monitoring system for a person includes one ormore wireless nodes forming a wireless network; a wearable sensor havinga wireless transceiver adapted to communicate with the one or morewireless nodes; and a software module receiving data from the wirelessnodes to detect changes in patient vital signs. In another aspect, amonitoring system includes one or more wireless nodes forming a wirelessnetwork; a wearable blood pressure sensor having a wireless transceiveradapted to communicate with the one or more wireless nodes; and asoftware module receiving data from the wireless nodes to detectdeteriorations in patient vital signs. In another aspect, a health caremonitoring system for a person includes one or more wireless nodesforming a wireless mesh network; a wearable appliance having a soundtransducer coupled to the wireless transceiver; and a bioelectricimpedance (BI) sensor coupled to the wireless mesh network tocommunicate BI data over the wireless mesh network. In another aspect, aheart monitoring system for a person includes one or more wireless nodesforming a wireless mesh network and a wearable appliance having a soundtransducer coupled to the wireless transceiver; and a heart diseaserecognizer coupled to the sound transducer to determine cardiovascularhealth and to transmit heart sound over the wireless mesh network to aremote listener if the recognizer identifies a cardiovascular problem.The heart sound being transmitted may be compressed to save transmissionbandwidth. In yet another aspect, a monitoring system for a personincludes one or more wireless nodes; and a wristwatch having a wirelesstransceiver adapted to communicate with the one or more wireless nodes;and an accelerometer to detect a dangerous condition and to generate awarning when the dangerous condition is detected. In yet another aspect,a monitoring system for a person includes one or more wireless nodesforming a wireless mesh network; and a wearable appliance having awireless transceiver adapted to communicate with the one or morewireless nodes; and a heartbeat detector coupled to the wirelesstransceiver. The system may also include an accelerometer to detect adangerous condition such as a falling condition and to generate awarning when the dangerous condition is detected. In yet another aspect,a monitoring system for a person includes one or more wireless nodesforming a wireless network; and a wearable device including: aprocessor; a transceiver coupled to the processor to communicate withthe one or more wireless nodes; a wearable sensor on a patch or bandagesecured to the person's skin and coupled to the processor; anaccelerometer coupled to the processor; and a thumb sensor coupled tothe processor. In another aspect, a health monitoring system for aperson includes a mobile telephone case including a cellular transceiverto provide wireless data and voice communication; a sensor including oneor more electrodes mounted on the mobile telephone case to contact theperson's skin and capture bio-electrical signals therefrom; an amplifiercoupled to the electrodes; a processor coupled to the amplifier; and ascreen coupled to the processor to display medical data such as imagesof the bio-electrical signals. Implementations of the above aspect mayinclude one or more of the following. The wristwatch determines positionbased on triangulation. The wristwatch determines position based on RFsignal strength and RF signal angle. A switch detects a confirmatorysignal from the person. The confirmatory signal includes a headmovement, a hand movement, or a mouth movement. The confirmatory signalincludes the person's voice. A processor in the system executes computerreadable code to transmit a help request to a remote computer. The codecan encrypt or scramble data for privacy. The processor can executevoice over IP (VOIP) code to allow a user and a remote person to audiblycommunicate with each other. The voice communication system can includeZigbee VOIP or Bluetooth or UWB VOIP or 802.XX VOIP. The remote personcan be a doctor, a nurse, a medical assistant, or a caregiver. Thesystem includes code to store and analyze patient information. Thepatient information includes medicine taking habits, eating and drinkinghabits, sleeping habits, or excise habits. A patient interface isprovided on a user computer for accessing information and the patientinterface includes in one implementation a touch screen; voice-activatedtext reading; and one touch telephone dialing. The processor can executecode to store and analyze information relating to the person'sambulation. A global positioning system (GPS) receiver can be used todetect movement and where the person falls. The system can include codeto map the person's location onto an area for viewing. The system caninclude one or more cameras positioned to capture three dimensional (3D)video of the patient; and a server coupled to the one or more cameras,the server executing code to detect a dangerous condition for thepatient based on the 3D video and allow a remote third party to viewimages of the patient when the dangerous condition is detected. Inanother aspect, a monitoring system for a person includes one or morewireless bases; and a cellular telephone having a wireless transceiveradapted to communicate with the one or more wireless bases; and anaccelerometer to detect a dangerous condition and to generate a warningwhen the dangerous condition is detected. In one aspect, systems andmethods include one or more entities including a sensor configured toprovide data in at least a first information standard from a firstmanufacturer and a second information standard from a secondmanufacturer; and an electronic health record database configured to:capture information from the one or more entities, normalize thecaptured information from first and second manufacturers in a commonformat, and add metadata for the captured information. In anotheraspect, an interoperable health-care system includes a network; one ormore medical data collection appliances coupled to the network, eachappliance transmitting data conforming to an interoperable format; and acomputer coupled to the network to store data for each individual inaccordance with the interoperable format. The user can take his/herweight, blood pressure, and cholesterol measurement daily, and the datais sent from a health base station to a monitoring service at hisdoctor's office. Periodically, the user gets an automated health summarygenerated by a service at his doctor's office as well as information tohelp him maintain a healthy lifestyle. The health information can bestored in an external HIPAA compliant health storage database so thatthe user and his doctor can access his health information over the web.The system extends health care system into the home and can recordpersonal health data on a systematic periodic basis. Appointments can beautomatically scheduled with providers. Long-term data for medicalbaseline can be collected. The system can also provide predictive alertsfor high-risk conditions. The system can perform initial triageutilizing biosensors, images, e-mail/chat/video.

In one embodiment, the radio is a micro-positioning radio such as a 5Genabled micro-positioning radio. IOT modules include a computerprocessor connected to UWB via either a cable or via a socketconnection. The modules also include a communication radio to send datato a separate processor for display. Modules can be placed on cornersbut can also be in a variety of components or added as a plug and playusing magnets or other forms of temporary attachments. The modules canbe placed on a support structure such as a room or a vehicle in atemporary fashion without manually measuring the position because theUWB can be used to range between modules and establish the room, officevehicle, lab, conference room, or cubicle as a constellation with knownrelative positions. The ranges between the modules are inputted to thesoftware on the processor. The software uses the ranges to create aknown geometric constellation of the UWB radios and then uses the knownoffset of the modules to calculate the relative locations of the modulesto one another. These ranges are then used by the software on theprocessor to trilaterate to the external device. The relative locationof the external device is used by software on the processor to produce arange and bearing to the potential target. Event Horizon Calculation isthen done. The range and bearing are inputted to software that isrunning a main event loop to track the event horizon—the timingassociated with a possible collision. The software stores the data in alinked list and uses this linked list to compare the current range andbearing to the previous range and bearing for that same external device.The distance between the current and previous locations is used tocalculate rate of speed and the time associated with nearby people andthen the radio ID of the nearby people can be recorded to enableaccurate and rapid automated contact tracing. In this manner, contacttracing using mobile app, smart watches, and physical tracing isprovided to rapidly contain infections. One embodiment provides a UWBExposure Notification Service for proximity detection of nearby wearabledevices and smartphones, and for the data exchange mechanism. ExposureNotification Service uses the UWB service for detecting deviceproximity. It uses a Temporary Exposure Key—A key that's generated every24 hours for privacy consideration. The result is a Diagnosis Key—Thesubset of Temporary Exposure Keys uploaded when the device owner isdiagnosed as positive for the coronavirus. A Rolling ProximityIdentifier which is a privacy preserving identifier derived from theTemporary Exposure Key can be sent in the broadcast of the UWB payload.The identifier changes about every 15 minutes to prevent wirelesstracking of the device. An Associated Encrypted Metadata (AEM) is aprivacy preserving encrypted metadata used to carry protocol versioningand transmit (Tx) power for better distance approximation. TheAssociated Encrypted Metadata changes about every 15 minutes, at thesame cadence as the Rolling Proximity Identifier, to prevent wirelesstracking of the device.

Another embodiment provides a smartphone app for employers that uses UWBsignals (but Bluetooth can be used as well), Wi-Fi, GPS and other datato track where employees go around the office, who they come intocontact with and for how long, to enable human resources or corporatesecurity managers to quickly access the data in the event of a workplaceoutbreak and notify employees who may have been exposed. Employees willwear wristbands or carry credit card-size badges that collect UWBsignals about their whereabouts and proximity to one another; that datais sent to devices that transmit it to the cloud. The chatbot identifiesspots where infected workers may have recently gathered, enablingcompanies to shut down specific areas, rather than an entire building,for deep cleaning. The badges are preferred where employees are notallowed to bring their personal phones, as well as to people who wouldrather not have their employers track them on their smartphones. AHealth Dashboard allows HR admins to view a list of their activeemployees, the most recent COVID-19 health status for each employee, andthe date the record was last updated. Admins can view more details abouteach employee's COVID-19 history (such as a list of test results overtime), and can click to verify or re-verify an employee's status. Whenan employer clicks “Verify”, the designated employee will receive acommunication such as a text or an email taking them to a consent-basedchatbot flow where they can securely share their COVID-19 health datawith their HR team. The employee will be required to submit informationsuch as recent lab test results, and the system may then verify thatinformation with the lab itself. The employee can connect tracing appsto the platform, confirming they have not been in contact with aninfected individual.

Office management will opt for the screening of all employees, vendorsand visitors entering their facility based on the most appropriatemethods for their particular space. These may include app-enabledquestionnaires, temperature checks, newly installed thermal cameras ordirect virus testing when it becomes more widely available. Hourly ordaily screenings of employees, vendors and visitors, making itcommonplace and fully integrated with the security access control systemto screen out people presenting with symptoms or known to be infected.As the availability of testing increases, those carrying antibodies ortesting negative for the virus will screen in and be allowed access.Lobbies include testing stations, screening queues, speed lanes,designated check in times and self-check kiosks. A building accesscontrol system is used as part of the contact tracing by mandatingcredential use for both entry and exit traffic for buildings, floors,tenant office suites and common areas at all times. UWB proximity datacan be further supplemented via intelligent face recognition learningmachines to investigate close personal contact for more detailedtracking so people who are impacted by pathogen exposure can be quicklyand easily notified. The chatbot can monitor and manage real-time spaceoccupancy, supplementing physical guides to reinforce social distancingwith real-time data reporting to provide notifications for issues suchas exceeding floor-level occupancy and suggest the need for greatersocial distancing if the number of people in a space is too high.Utilizing access control, the chatbot can assist tenants in enforcingstaggered work schedules to minimize density.

By enabling a network of readily connected health and medical devices,people with Covid or infectious disease or other chronic diseases willbe able to share vital sign information such as blood pressure andglucose level with their doctors. Adult children will be able toremotely watch over their aging parents and proactively help them managesafely in their own homes. Diet and fitness conscious individuals willalso be able to seamlessly share their weight and exercise data withfitness consultants through the Internet. The above system forms aninteroperable health-care system with a network; a first medicalappliance to capture a first vital information and coupled to thenetwork, the first medical appliance transmitting the first vitalinformation conforming to an interoperable format; and a second medicalappliance to capture a second vital information and coupled to thenetwork, the second medical appliance converting the first vitalinformation in accordance with the interoperable format and processingthe first and second vital information, the second medical applianceproviding an output conforming to the interoperable format. Theappliances can communicate data conforming to the interoperable formatover one of: cellular protocol, ZigBee protocol, Bluetooth protocol,WiFi protocol, WiMAX protocol, USB protocol, ultrawideband (UWB)protocol. UWB is a short-range, wireless communication protocol thatuses a wide spectrum of several GHz. UWB acts as a radar that cancontinuously scan an entire room and precisely lock onto another UWBobject or mobile device to discover its location and communicate dataand for location discovery and device ranging with precision. Theappliances can communicate over two or more protocols. The first medicalappliance can transmit the first vital information over a first protocol(such as Bluetooth or UWB protocol) to a computer, wherein the computertransmits the first vital information to the second medical applianceover a second protocol (such as ZigBee prototocol). The computer canthen transmit to a hospital or physician office using broadband such asWiMAX protocol or cellular protocol. The computer can perform theinteroperable format conversion for the appliances or devices, oralternatively each appliance or device can perform the formatconversion. Regardless of which device performs the protocol conversionand format conversion, the user does not need to know about theunderlying format or protocol in order to use the appliances. The useronly needs to plug an appliance into the network, the data transfer isdone automatically so that the electronic “plumbing” is not apparent tothe user. In this way, the user is shielded from the complexitysupporting interoperability. In another aspect, a monitoring system fora person includes one or more wireless nodes and a stroke sensor coupledto the person and the wireless nodes to determine a medical problem, forexample a stroke attack. The stroke monitoring system is interoperablewith emergency vehicle and/or hospital systems and provides informationto quickly treat stroke once the patient reaches the treatment center.

In one aspect, a monitoring system for a person includes one or morewireless nodes and an electromyography (EMG) sensor coupled to theperson and the wireless nodes to determine a medical issue such as astroke attack. In another aspect, a health care monitoring system for aperson includes one or more wireless nodes forming a wireless meshnetwork; a wearable appliance having a sound transducer coupled to thewireless transceiver; and a bioelectric impedance (BI) sensor coupled tothe wireless mesh network to communicate BI data over the wireless meshnetwork. In a further aspect, a heart monitoring system for a personincludes one or more wireless nodes forming a wireless mesh network anda wearable appliance having a sound transducer coupled to the wirelesstransceiver; and a heart disease recognizer coupled to the soundtransducer to determine cardiovascular health and to transmit heartsound over the wireless mesh network to a remote listener if therecognizer identifies a cardiovascular problem. The heart sound beingtransmitted may be compressed to save transmission bandwidth. In yetanother aspect, a monitoring system for a person includes one or morewireless nodes; and a wristwatch having a wireless transceiver adaptedto communicate with the one or more wireless nodes; and an accelerometerto detect a dangerous condition and to generate a warning when thedangerous condition is detected. In yet another aspect, a monitoringsystem for a person includes one or more wireless nodes forming awireless mesh network; and a wearable appliance having a wirelesstransceiver adapted to communicate with the one or more wireless nodes;and a heartbeat detector coupled to the wireless transceiver. The systemmay also include an accelerometer to detect a dangerous condition suchas a falling condition and to generate a warning when the dangerouscondition is detected. Implementations of the above aspect may includeone or more of the following. The wristwatch determines position basedon triangulation. The wristwatch determines position based on RF signalstrength and RF signal angle. A switch detects a confirmatory signalfrom the person. The confirmatory signal includes a head movement, ahand movement, or a mouth movement. The confirmatory signal includes theperson's voice. A processor in the system executes computer readablecode to transmit a help request to a remote computer. The code canencrypt or scramble data for privacy. The processor can execute voiceover IP (VOIP) code to allow a user and a remote person to audiblycommunicate with each other. The voice communication system can includeZigbee VOIP or Bluetooth or UWB VOIP or 802.XX VOIP. The remote personcan be a doctor, a nurse, a medical assistant, or a caregiver. Thesystem includes code to store and analyze patient information. Thepatient information includes medicine taking habits, eating and drinkinghabits, sleeping habits, or excise habits. A patient interface isprovided on a user computer for accessing information and the patientinterface includes in one implementation a touch screen; voice-activatedtext reading; and one touch telephone dialing. The processor can executecode to store and analyze information relating to the person'sambulation. A global positioning system (GPS) receiver can be used todetect movement and where the person falls. The system can include codeto map the person's location onto an area for viewing. The system caninclude one or more cameras positioned to capture three dimensional (3D)video of the patient; and a server coupled to the one or more cameras,the server executing code to detect a dangerous condition for thepatient based on the 3D video and allow a remote third party to viewimages of the patient when the dangerous condition is detected. Moredetails are disclosed in application Ser. No. 16/894,040 and 16/894,058,the contents of which are incorporated by reference.

In one aspect, a method to assist people in an infectious diseaseoutbreak includes: providing a mobile fitness device to monitor andupload activity and vital sign to a fitness device server on a periodicbasis; collecting daily health data from the fitness device server,collecting medical data for a person from a medical institution, andcollecting infectious treatment data from predetermined sourcesincluding a government and a non-governmental organization (NGO);training a chatbot with a deep neural network with the collected data;and responding to a query by querying data from the mobile fitnessdevice and retrieving an answer from the deep neural network based onvital sign and activity history. In another aspect, a system includes amobile fitness device to collect exercise and vital signs from a user;and a chatbot to assist people in an infectious disease outbreak with aprocessor to: provide a mobile fitness device to monitor and uploadactivity and vital sign to a fitness device server on a periodic basis;collect daily health data from the fitness device server, collectingmedical data for a person from a medical institution, and collectinginfectious treatment data from predetermined sources including agovernment and a non-governmental organization (NGO); train a chatbotwith a deep neural network with the collected data; and respond to aquery by querying data from the mobile fitness device and retrieving ananswer from the deep neural network based on vital sign and activityhistory.

Implementations of the above aspects may include one or more of thefollowing additions to the above aspect:

2. training the deep learning machine with logically grouped orclustered data to provide context and biasing the answer based on thecontext.

3. grouping the data by age, sex, race, home location, health history,exercise history, family genetics, social economics, or risks for one ormore diseases.

4. collecting recent data from mobile fitness devices, retrieving priorinteractions with the user and prior health reports, and history ofpeople in contact with the user.

5. collecting contract tracing data and training the deep neural networkwith the contact tracing data and data from people having contacts witha user.

6. capturing contract tracing data using ultra-wide-band (UWB).

7. determining a group or cluster best matching the person's healthcondition data and apply the customization information to bias thelearning machine to generate a context-sensitive answer.

8. detect a user emotion during a chat and altering the answer based onthe user emotion to provide empathy or to request professional help.

9. detecting emotion using a user facial expression or a verbalexpression.

10. detects risks including suppressed immune system, a cancercondition, an organ transplant condition, nfectious diseasesusceptibility, a healthcare work condition, an at-risk location.

11. collecting vital signs from mobile fitness device and detecting acore body temperature pattern, breathing pattern, coughing pattern, andwalking/exercise patterns to detect changes indicative of an infectiousdisease.

12. detecting a breathing rate, a coughing pattern, a walking pattern,an exercise pattern.

13. detecting with the chatbot and the mobile fitness device fever orchills, cough, shortness of breath or difficulty breathing, fatigue,muscle or body aches, Headache, New loss of taste or smell, sore throat,congestion or runny nose, nausea or vomiting, diarrhea, breathingproblem, chest pain or pressure, confusion during chat, ability to wakeor stay awake, color change in a lip or a face.

14. detecting if a chronic condition needs treatment, and recommendingtreatment when treatment is suspended.

15. providing a rich text, a structured markup, a schema, microdata, orsematic tags on a web page for search engine optimization.

16. providing a Claim Review Schema on a web page to improve trust in adisease recommendation.

In another aspect, a chatbot can be used for determining an infectioncandidate, comprising: receiving COVID trial enrollment criteria from auser including a combination of genetic variants for which a drug ortherapy is likely to respond; searching a knowledge base of patient testinformation received from a plurality of independent entities forpatients that match the trial enrollment criteria, wherein the knowledgebase comprises an ontology data structure that identifies a causalrelationship between a genetic variant and a phenotype based on acombination of the genetic variant and modifier variant information,wherein the knowledge base links the genetic variant and the modifiervariant information, wherein the modifier variant information is basedon curated evidence, and wherein the modifier variant informationidentifies whether modifier variants that modify a severity of thephenotype are likely to exist; and providing to the user search resultsfor consented patients that match the trial enrollment criteria; whereinat least one of the receiving, searching, or providing are performed byone or more computers. In implementations, the test informationcomprises at least one of patient test information, patient sequencevariant information, patient medical record information, patientlocation information, test site location information, patient phenotypeinformation, and patient consent information.

Advantages of the medical chatbot includes one or more of the following.The bot enables healthcare companies and government entities to reachpatients and audience directly. The bot answers questions in a realisticand with empathy through engaging use of personality, knowledge anddisplay of empathy. The length of the agent's utterances is important inachieving better results with human evaluators. If they're too short,the responses are dull and communicate a lack of interest; if they'retoo long, the chatbot seems to waffle and not listen. The bots also usereal time data from fitness monitoring devices such as smart watch andincorporate that information into the conversation and get timelyassistance or care for the patient. The bot helps agencies save time andmoney on patient care. Patients and customers expect 24/7 availability,but they hate waiting on hold. They also ask many of the same questionsover and over (and over) again. The bots greet potential customers, byidentifying their needs, asking basic questions, (i.e., “Do you have anysymptoms?”) and only direct urgent care issues to medical professionals.Bots can do the selling with the right script. With the bot,conversational commerce leaves room for personalized upselling as thebot makes suggestions. The bots are capable of retaining information,and those details can reach out personally, offering relevant content atthe right time. The bot reminds employees to apply the hand sanitizerand cleaning wipes on the premises, and checks that their offices all becleaned and sanitized frequently. The system reminds employees atappropriate time to conform to Social Distancing. Data from chatbotscreening enables employees to feel comfortable coming back to theiroffices. A person who is known to be infected or showing symptoms is notto be allowed access. The system provides Privacy & Security. Like othersensitive healthcare information, COVID-19 status data needs to behandled extremely carefully by employers. Top of mind considerationsinclude receiving consent from employees to share their healthinformation, securing data infrastructure to store this info, andlimiting access control to this information within the company. Thesystem provides flexibility for multiple inputs. Guidance fromgovernment agencies and the scientific community is changing all thetime on what an employer needs to verify to bring an employee back towork. This includes a combination of antibody test results, data pulledfrom contact tracing apps, and any history of confirmed infection. HRteams need to connect these inputs to a variety of human resourcesinformation systems (HRIS) to match active employee records withCOVID-19 related data. With people consent, the system can combinedifferent inputs (such as user-submitted information and trustedthird-party sources) to verify this sensitive data while restrictingaccess to employees and designated admins.

FIG. 4A shows top level views of the GPT, BERT, and Transformerarchitectures with a token bias process to provide context sensitiveshort or long form text generation. The context sensitivity becomesimportant in long form text generation as the result is more responsiveto the brief text provided by the user who expects the system to amplifyhis/her thoughts into a full sentence or paragraph in the case ofdrafting text. In the case of code generation or ASIC chip generation,the token bias allows more accurate functional blocks to be suggested ina top-down design system, for example.

The GPT-2 is built using transformer decoder blocks. The model isconstructed using the basic concept of Transformer, Attention, etc, forpre-training a dataset composed of Common Crawl, Wikipedia, WebText,Books and some additional data sources. The GPT-3 language model has 175billion parameters. A parameter is a measurement in a neural networkthat deploys a large or small weightage to a few aspects of data, forproviding that aspect larger or smaller importance in an entiremeasurement of the data. These are the weights that deliver shape to thedata, and provide a neural network an understanding angle on the data.GPT-3 involves adjusted initialization, pre-normalization, andchangeable tokenization. It reflects substantial performance on variousNLP tasks and benchmarks in three distinct shots, i.e. zero-shot,one-shot and some-shot environments. BERT, on the other hand, usestransformer encoder blocks. One difference between the two is that GPT2,like traditional language models, outputs one token at a time, the modelto predict the next token in a sequence, rather than converting onesequence to another functionally identical one. The output layer ismodified to reflect the probability biasing. These models predict thenext token in a sequence, rather than converting one sequence to anotherfunctionally identical one, and the output layer uses probabilitybiasing discussed above. FIG. 4B shows the encoder and decoder stacks ofthe Transformer architecture. FIG. 4C shows in more detail the encoderand decoder blocks of the Transformer architecture with the outputprobabilities biased to account for context in generating machineresponses.

FIGS. 4D-4E show additional views of the transformer architecture thattakes a sequence of n word embeddings. For position information, apositional embedding is added to each word embedding using sine andcosine functions to form a continuous binary encoding of positions in asequence. Multihead attention is used to encode the input embeddingswhere input order in the sequence is lost so positional embeddings areused. As is known to one skilled in the art, the transformer uses theencoder attention, the encoder-decoder attention and the decoderattention. The attention mechanism is implemented as a vectormultiplication, where the angle of the vector one can determine theimportance of each value. If the angles of the vectors are close to 90degrees, then the dot product will be close to zero, but if the vectorspoint to the same direction, the dot product will return a greatervalue. Each key has a value associated, and for every new input vector,we can determine how much does this vector relates to the value vectors,and select the closest term using a softmax function. Transformers havea multihead attention; similar to filters in CNN's, each one learns topay attention to a specific group of words. One can learn to identifyshort-range dependencies while others learn to identify long-rangedependencies. The model to predict the next token in a sequence, ratherthan converting one sequence to another functionally identical one.There would also be some changes made to output layer (probabilitybiasing). This improves the context-awareness to help the modeldetermine the terms referred to when it's not clear; for example, withwords such as pronouns.

The Encoder and Decoder are composed of modules that can be stacked ontop of each other multiple times and the modules consist mainly ofMulti-Head Attention and Feed Forward layers. The inputs and outputs(target sentences) are first embedded into an n-dimensional space sincestrings are not used directly. The positional encoding of the differentwords are added to the embedded representation (n-dimensional vector) ofeach word. One commonly used attention calculation can be:

${{Attention}\left( {Q,K,V} \right)} = {{{softmax}\left( \frac{QK^{T}}{\sqrt{d_{k}}} \right)}V}$

where Q is a matrix that contains the query (vector representation ofone word in the sequence), K are all the keys (vector representations ofall the words in the sequence) and V are the values, which are again thevector representations of all the words in the sequence. For the encoderand decoder, multi-head attention modules, V consists of the same wordsequence than Q. However, for the attention module that is consideringthe encoder and the decoder sequences, V is different from the sequencerepresented by Q. To simplify, the values in V are multiplied and summedwith attention-weights a, defined by:

$a = {{softmax}\left( \frac{QK^{T}}{\sqrt{d_{k}}} \right)}$

Weights a are defined by how each word of the sequence (represented byQ) is influenced by all the other words in the sequence (represented byK). Additionally, the SoftMax function is applied to the weights a tohave a distribution between 0 and 1. Those weights are then applied toall the words in the sequence that are introduced in V (same vectorsthan Q for encoder and decoder but different for the module that hasencoder and decoder inputs).

The attention-mechanism can be parallelized into multiple modules and isrepeated multiple times with linear projections of Q, K and V. Thisallows the system to learn from different representations of Q, K and V.These linear representations are done by multiplying Q K and V by weightmatrices W that are learned during the training. Those matrices Q, K andV are different for each position of the attention modules in thestructure depending on whether they are in the encoder, decoder orin-between encoder and decoder. The reason is that we want to attend oneither the whole encoder input sequence or a part of the decoder inputsequence. The multi-head attention module that connects the encoder anddecoder will make sure that the encoder input-sequence is consideredtogether with the decoder input-sequence up to a given position. Afterthe multi-attention heads in both the encoder and decoder, thetransformer has a pointwise feed-forward layer. This feed-forwardnetwork has identical parameters for each position, which can bedescribed as a separate, identical linear transformation of each elementfrom the given sequence.

While the system uses a standard transformer as described above, theprocess applies the above commonly used transformer architecture andtunes the training for long text generation that is guided by an outlineso that the long form text is useful. This combines increased model sizewhile sacrificing convergence by stopping training early. As largermodels converge to lower test error in fewer gradient updates thansmaller models, large models achieve higher accuracy faster for trainingand speed during inference is achieved using model compression. In theinstant process, large models are used on large text clustered intospecific groups or technology or market segments, or IPC code, forexample. The output probabilities are biased according to acustomization indicium data (for example the IPC mentioned above). Suchtraining creates custom models for each context based on the outputprobabilities as biased. One embodiment uses quantization and pruning toreduce the inference latency and memory requirements of storing modelweights. Quantization stores model weights in low precision and pruningsets predetermined NN weights to zero.

During inference, the process includes:

-   -   Input the full encoder sequence (a short phrase) and as decoder        input, an empty sequence is used with only a start-of-sentence        token on the first position. This will output a sequence with        the first element.    -   That element will be filled into second position of the decoder        input sequence, which now has a start-of-sentence token and a        first word/character in it.    -   Input both the encoder sequence and the new decoder sequence        into the model with the biased output probability incorporating        the context sensitive data. Take the second element of the        output and put it into the decoder input sequence.    -   Repeat this until done.

One embodiment predicts an end-of-sentence token, which marks the end ofthe phrase expansion into a sentence, paragraph, or long form text,among others.

Multiple runs through the model are used for the text expansion process.

The models can have different parameters of the Transformer, such as thenumber of decoder and encoder layers, and the results can be tuned andtrained with large corpus for improving output.

In another embodiment for video inferencing, the process is trained onpredicting an image (or brief video) and generating a longer videosequence. The process includes:

-   -   Input the full encoder sequence (a short phrase or starting        image/video) and as decoder input, an empty sequence is used        with only a start-of-video token on the first position. This        will output a sequence with the first element.    -   That element will be filled into second position of the decoder        input sequence, which now has a start-of-video token and a first        image in it.    -   Input both the encoder sequence and the new decoder sequence        into the model (optionally with the biased output probability        incorporating the context sensitive data in another embodiment).        Take the second element of the output and put it into the        decoder input sequence.    -   Repeat this until done.

Multiple runs through the model are used for the video expansionprocess.

One embodiment generates videos from a milestone image. They can usetransformers, GANs, and VAEs, or combinations thereof. One embodiment(FIG. 4H) uses Generative Adversarial Network (GAN), a framework fortraining generative models in an adversarial setup with two networks, agenerator that creates object instances (e.g., images, sentences) andtries to fool a discriminator; and a discriminator is trained todiscriminate between real and synthetic object instances.

FIG. 4G shows a convolutional network for generating videos fromthumbnail images or videos in storyboards. The input clip goes through aseries convolutions and nonlinearities that preserve resolution. Afterintegrating information across multiple input frames (if multiple), thenetwork up-samples temporally. The network outputs codes for atransformation of the input frames, which produces the final video. Inthe transformations: For each (x; y; t) coordinate in the videoexpansion, the network estimates a weighted combination of neighboringpixels from the input frame to render the predicted frame. Thetransformation is applied by convolution. The transformer outputprobability is biased by video context as done in the priortransformers.

One embodiment uses the GAN with a spatio-temporal convolutionalarchitecture that untangles the scene's foreground from the background.This model can generate tiny videos up to a second at full frame ratebetter than simple baselines and can predict plausible futures of staticimages. The generator uses a deep convolutional network that inputslow-dimensional random noise and outputs a video. Spatiotemporalup-convolutions (2D for space, 1D for time) are used to model video. Thegenerator also models the background separately from the foreground. Thenetwork produces a static background (which is replicated over time) anda moving foreground that is combined using a mask. A discriminatornetwork is used to distinguish real videos from fake videos.

Another embodiment utilizes GANs with Spatial Transformer Networks(STNs) as the generator or Spatial Transformer GANs (ST-GANs). ST-GANsseek image realism by operating in the geometric warp parameter space.The ST-GAN can generate high-resolution images indirectly since thepredicted warp parameters are transferable between reference frames.

Yet another embodiment uses Variational Autoencoders (VAEs) with twoneural networks: an encoder comprised of convolutional layers thatencode an object (image, text, sound) into a latent vector; and adecoder comprised of deconvolutional layers that decode a latent vectorback into the object. As the autoencoder network reconstructs the databut cannot generate new objects, the variational autoencoder (VAE)requires an additional feature that allows it to learn the latentrepresentations of the inputs as soft ellipsoidal regions rather thanisolated data points. New data can be generated by sampling latentvectors from the latent space and passing them into the decoder.

The system can help expand the creativity of the user, and oneembodiment applies the system for educational or other charitablepurposes. Next, a method for financing education for a student studyingat an institution by leveraging from the student's creativity includes:providing an on-line creative work generation tool to the student todraft and submit a creative work as part of an entrance requirement, aclass requirement or a graduation requirement; and receiving a completedcreative work and checking creative work quality and upon passingacceptant criteria, rewarding the student or the institution with agrant to offset educational expenses for the student.

The above method includes one or more of the following implementationdetails:

-   -   Crowd-sourced quality assurance of the student's creative work        by having other students in the class rate, review and critique        the creative work.    -   Crowd-sourced quality assurance of the student's creative work        by having the teacher or professor for the class rate, review        and critique the creative work.    -   Crowd-sourced quality assurance of the student's creative work        by having an intra school competition where selected members of        the school rate, peer-review and critique the creative work.    -   Crowd-sourced quality assurance of the student's creative work        by having an inter-school competition where judges rate, review        and critique the creative work.    -   Crowd-sourced quality assurance of the student's creative work        by having industry-experts rate, review and critique the        creative work.    -   Crowd-sourced quality assurance of the student's creative work        by having a creative work searcher rate, review and critique the        creative work.    -   The system can rank the quality of the creative work by        comparing the requested exclusivity to a library of white-spaces        or open spaces for creative working.    -   The school can require each student to submit a creative work as        part of the graduation requirement.    -   Criteria for grading the creative work can include detail/depth        of solution based on number of figures/pages.    -   The system can establish a professional internship program by        obtaining a sponsor contract with a sponsor for a full-time        equivalent job internship position to commercialize the subject        of the creative work; establishing a program by obtaining a        license agreement for the creative work; and acquiring donations        for the creative work.    -   The individual funds are tuition, vouchers, public vouchers,        private vouchers, grants, and scholarships, government funds,        charter funds.    -   The educational institution can be physical locations or can be        virtual schools where students attend class over the Internet.    -   The sponsor can be a business, the school, a not-for-profit        entity, or a government entity.    -   A college incentive program can be formed by providing the        students direct subsidies for college or college loan repayment        assistance upon sale of the creative work.    -   The student can form a venture to commercialize the idea. The        venture can be funded by investors including alumni of the        institution, angels, venture capitalists, crowdfunding,        micro-funds, or microloans.    -   The institution can have a financial stake in the new venture in        exchange for use of its assets, such as athletic facilities,        classrooms, laboratories, libraries, media centers, fine art        spaces, performing arts spaces, conference room, technology        centers, and the brand name of the institution.    -   The funders can base their decisions on the novelty of the idea,        the current commercial trends in the creative workspace        associated with the creative work.    -   The funders can base their decisions on the students' financial        history, grade or educational ranking, and teacher        recommendation.    -   The funders can purchase the creative work and bundle the        application into bundles for sale or for use to license as a        defense or counterclaim in a creative work litigation.

FIG. 5A shows one embodiment for enabling more students to attend schooland providing more resources for the school. The process includesproviding an on-line creative work generation tool to the student todraft and submit a creative work as part of a class requirement orgraduation requirement (50) and receiving a completed creative work andchecking creative work quality and upon passing acceptant criteria,rewarding the student or the institution to offset educational expensesfor the student (60).

FIG. 5B shows another embodiment for non-profit financing using studentcreativity. The method includes providing an on-line creative workgeneration tool to the student to draft and submit a creative work aspart of a class requirement or graduation requirement. Crowd-sourcedquality assurance can be done for the student's creative work by havingother students in the class rate, review and critique the creative work.Other people who performs quality assurance for creative work caninclude the teacher or professor or industry expert who rate, review andcritique the creative work. The system receives a completed creativework and checking creative work quality and upon passing acceptantcriteria, rewards the student or the institution to offset educationalexpenses for the student. To generate funds for the financing process,the system can pool the creative works to bundles of rights forcommercialization with companies.

In one embodiment, when the student submits the creative work as part ofthe entrance or graduation requirement, the system checks forplagiarism. If plagiarism is detected, the submission is rejected, andschool ethics officials are notified and the student faces aninvestigation and the penalty associated with cheating. Moreover, thestudent's file is annotated on the social network profile and the creditrating for microloans is negatively affected. The plagiarism checkincludes checking the creative work that has been submitted as anindividual's own work against creative work database and a searchengine.

One tool used to help grade the submission and rate the quality of thecreative work or intangible asset employs information retrievaltechnique and/or a learning machine that examines the text of a set ofexclusionary claims or requested exclusivity that defines, in technicalterms, the extent, i.e. the scope, of the protection sought in anapplication to be submitted to an authority such as a government agencyfor example. In other words, the purpose of the claim or exclusivity isto define which subject-matter is protected upon issuance of thegovernment grant. This is termed as the “notice function” of theexclusivity or claim—to warn others of what they must not do or copy.The corresponding creative works or intangible asset may then be rankedaccording to the degree to which their respective requested exclusivitysets represent significant innovation above and beyond existing work.For example, a creative work may be considered valuable if the subjectmatter in the creative work is cited by, relied upon, or expanded uponin subsequently filed creative works. In preferred embodiments andimplementations, a user may interact in the process to refine theanalysis.

Another embodiment of a system or method of financing or providingeducation utilizes an extension program for alumni to use the IPdevelopment program to leverage the connection with the school to getfunding for a business idea. In an embodiment, the extension program isa subsidiary of the school, structured as a charitable organization. Inone embodiment, each accepted creative work earns the student points inan account managed by the school. When the student has earned sufficientpoints, he or she can redeem the points at the school web site to assistthe student or one or more other students to pay for higher-educationcosts such as books and other supplies.

In another embodiment, inventors not affiliated with the school candonate ideas and creative work to the school by submitting the conceptsusing the creative work generation system as donations and then candesignate the resulting awards to be disbursed to students. The schoolacts as an escrow for the future monetization of the idea, and theschool can provide a tax receipt showing a predetermined value for theidea based on market valuation so that the donor can get tax benefitsand when future royalties arrive, the university can provide taxdonation receipts to the donor using predetermined formulas. Thus, anymonetization of such donated work will be credited to the donor in theform of tax benefits and good will as donors to the university, whileall the rewards accrue for the benefit of the higher-educationinstitution or direct to the students.

In another embodiment, an entrepreneur student can apply formicro-financing to move his/her idea to the marketplace. The student canalso borrow small amounts to finance educational expenses. Themicrolender can make quick decisions from big data associated with cellphone and grade point information, among others. For example, cell phoneusage can provide:

Average prepaid balance for each of the last twelve months List of alltop up transactions (volume, date) Top up regularity and frequency willbe calculated from #3 Incoming call volume Outgoing call volume Numberof unique incoming calls (ie different numbers) Number of uniqueoutgoing calls (ie different numbers) Geographical reach of incomingcalls Geographical reach of outgoing calls Total number of incomingtexts Total number of outgoing texts Distance between furthest points ofthe user's locations Total miles unveiled Average length of incomingcalls (exclude promotional ones unless everyone gets the same ones),Average length of outgoing calls Total number of minutes the phone isconnected to the network Total amount of time in use ie percentageactivity metric for all activities Texts coming from unique numbersTexts going to unique numbers Call regularity Call frequency Averageprepay balance when new airtime is purchased Number of SIM cards forgiven mobile operator Length of time SIM card has been owned Whether theSIM has been transferred to or from a different mobile operator GenderZip code of SIM registration could also be useful to compare to zip codeof loan app) Age/Date of Birth Active mobile money account Any paymentsto businesses/institutions (and volume, frequency, etc.) Average balancein mobile money account Number of missed calls per month Data usage permonth Spending on extras (ringtones, etc) Tariff/rate plan changes inpast X months Number of international calls incoming/outgoing

In another embodiment, micro-lending for a small amount such as $5 ormore can be done by the system upon receipt of the creative worksubmission. The system can rate the creative work as detailed above, andadditionally, the student credit rating can be inferred from his/hercell phone statistics, and grade point average and other data gleanedfrom the students' social network activity. Based on the information, aregression can be done to estimate the probability of defaults so thatthe computer can allocate lending for educational use or for launching aventure based on the creative work concepts.

Other embodiments provide a method of financing expected futureeducational expenses by (a) calculating future educational expensesbased on current educational expenses, past changes in educationalexpenses, and assumptions on annual increase rates of educationalexpenses; (b) establishing a target for future total investment payoutfor total future educational expenses based on calculated futureeducational expenses, and assumptions on expected investment yields; (c)deriving present investment amount needed to provide future educationalexpenses; (d) generating creative works or intangible assets to sell tosatisfy the present investment amount and collecting payment of thepresent investment amount; and (e) investing the payment to providefunds for payment of the future educational expenses.

In another embodiment shown in FIG. 5C, the non-profit entity can poolthe ideas into pools that can be used to license commercial entities whoin turn pay into a pool to support the non-profit entity. The pool caninclude unsolicited new ideas and solicited ideas requested bycompanies, for example. In one example, students can come up withingenous ideas on a problem without any prompting. In another example, acompany can seek ideas in a particular task. For example, SBIR programscommonly request solutions to a predetermined problem, and theuniversity can put these requests on a “problem to be solved” page wherestudents can propose concepts for professor review and upon agreementbetween professors and students, the students can be granted access tothe instant system to document their ideas before implementation and toupdate the ideas as they are implemented. Moreover, students in the sameschool or entity can join the system and collaborate as co-creators ofthe idea. A pool of ideas can be created and such pools can be offeredto companies for use. This is controlled crow-sourcing where thecompanies gain benefit from a large number of brains and external ideaswithout a large risk of paying for a large R&D department that may haveNIH syndrome, for example. In one implementation, the pools canperiodically apply as provisionals using university discounts ornon-profit discounts, and immediately available under agreements to thecompanies to try for one year, and one utility conversion that is keptalive through continuations is done to ensure continuity of protectionfor all ideas in the provisional document. This arrangement keeps largenumber of ideas in the pool alive for twenty years so the university ornon-profit can continue to benefit while keeping cost low. Companiesthat receives rights from the university or non-profit also can claimthe benefit of exclusivity or the good will of donors to the schools, orboth.

In another embodiment, a company can make available the system as afreemium system where users can try the system for free up to apredetermined number of drawings such as 3 figures. After that, thesystem offers continued usage for a fee or for a percentage of profit,for example. Ideas entered using the system can be securely trackedusing Ethereum blockchain as detailed in U.S. patent Ser. No. 10/195,513to the instant inventor, the content of which is incorporated byreference. The use of blockchain provides solid proof of conception andownership in case the priority date is important, and the use of theinstant methods enable rapid filing in a first to file system. Theblockchain annotation also is proof that the system was used to generatethe idea at a particular time to resolve any rights disputes.

In another embodiment, the system can be ad-supported. As new productsare being generated, marketers, consultants, developers can offer toprovide commercialization services to the user of the system or portal.As disclosed in United States Patent Application 20060190807 to theinstant inventor, the content of which is incorporated, after connectingto the portal, the assistant checks for the latest updates in his areasof Interest and show them in a small window at the bottom left portionof the screen. The client software performs multiple tasks, includingestablishing a connection to the portal; capturing demographicinformation; authenticating a user via a user ID and password; trackingWeb-sites visited; managing the display of advertising banners;targeting advertising based on Web-sites visited and on keyword search;logging the number of times an ad was shown and the number of times anad was clicked on; monitoring the quality of the online sessionincluding dial-up and network errors; providing a mechanism for customerfeedback; short-cut buttons to content sites; an information ticker forstocks, sports and news; and a new message indicator. When the useraccesses the portal, a background window is shown on his or her computerscreen that is always visible while the user is online, regardless ofwhere the user navigates. The window displays advertisements,advertiser-sponsored buttons, icons and drop-down menus. By clicking onitems in the background window, users can navigate directly to sites andservices such as news, laws, seminars and conferences, connections toothers with similar interests, auctions & exchanges, lawyers,businesses, mediators between two companies contesting the same IPsubject matter, forms such as a non-disclosure agreement, IP updates andmarket place updates. Revenues can be generated by sellingadvertisements and sponsorships on the background window and byreferring users to sponsors' Web-sites. The assistant showsadvertisements while its window is visible. If the user clicks on anadvertisement or news or related feature, the assistant willautomatically launch the browser and take the user to the advertiser'ssite. The portal incorporates data from multiple sources in multipleformats and organizes it into a single, easy-to-use menu. Information isprovided to the public free-of-charge with value added databases andservices such as patent drafting assistance available to subscribers whopay a subscription fee. At a first level, the public can use withoutcharge certain information domains in the portal. At a second level,individual inventors, very small companies and academic users can accessthe patent drafting software when they subscribe to a first plan with apredetermined annual membership fee and a transaction fee charged perpatent application. At a third level, companies can access additionalresources such as an IP portfolio management system, a docket managementsystem, a licensing management system, and a litigation managementsystem, for example. In this manner, the portal flexibly andcost-effectively serves a variety of needs. Other resources that theportal provides access to include traders who mediate between potentiallicensors and licensees.

The portal also provides access to a bid, auction and sale systemwherein the computer system establishes a virtual showroom whichdisplays the IPs offered for sale and certain other information, such asthe offeror's minimum opening bid price and bid cycle data which enablesthe potential purchaser or customer to view the IP asset, view ratinginformation regarding the IP asset and place a bid or a number of bidsto purchase the IP asset. The portal has access to IP search enginesthat continuously search the web and identify information that is ofinterest to its users. These search engines will use the user profilesto search the web and store the results in the user folders. Thisinformation is also relayed to the users using the assistant. The portaldelivers focused IP contents to interested subscribers and indirectlydrives these subscribers and their businesses to innovate. The portalthus allows users to draft their own applications rapidly and accuratelyand in a manner that conforms to the requirements of the major nationalpatent offices. Quality in the resulting patent application is achievedby providing an expert system in our software that guides membersthrough each step of preparing an application. Speed is achieved byintegrating the IP generation process with existing business workflow.When a communication from the patent office is received, the expertsystem guides the user through the process of responding. Since themember is generating the bulk of the work product, the cost in procuringthe IP asset is reduce, while responsiveness is enhanced. A network ofindependent professionals such as lawyers can perform value-addedpre-filing check to enhance the member's work product, if desired.Information relating to the network of attorneys will be maintained in asearchable database. Thus, members can search by the attorney's specificexpertise (legal as well as technical) and by location. Members can thenemail the selected attorney a question. To prevent conflict issues, themembers will be warned that the first question should be couchedabstractly so that the invention is not revealed. Further, each attorneyin the network automatically observes the applicable conflict rules inhis or her jurisdiction before taking on the question. One or moreattorneys in the network can respond to the first question to initiatethe consultation process, if no conflict exists. The parties can thennegotiate fees relating to subsequent questions and/or work. As such,the portal supports a market-based system for getting qualifiedassistance. The portal generates revenues by providing advertisementspace to law firms, attorneys, patent-support businesses andcorporations. By having access to the member's IP interests, the Website can provide pre-screened, high-quality investment opportunitiesthat match the investor's identified interests. The web site thus findsand adds value to potential deals, allows investors to invest from seedfinancing right through to the IPO, and facilitates the hand off to toptier underwriters for IPO. Additionally, members have access to a broadcommunity of investors focused on the cutting edge of high technology,enabling them to work together as they identify and qualify investmentopportunities for IP or other corporate assets.

In one embodiment, an incubator model can be used where the incubatorprovides access to the instant tools herein, and further provides accessto an ecosystem of investors and start-up consultants that can help theentrepreneur with engineering/development, sales/marketing, production,human resource, banker, lawyers, among others. In another embodiment,the system can be part of a crowdfunding platform where entrepreneurscan establish connection with investors via four different fundingoptions (smart contracts) offered on the marketplace (donation, debt,revenue share, and equity).

Investors who wish to support an entrepreneur's creative idea/projecthave the opportunity to purchase branded tokens backed by the IP createdusing the instant system. Investors can pay in any tokens/coins ordirect Ethereum blockchain transactions to buy the tokens. Theinvestor's contribution is sent to Vault (secured money storage withinsmart contract). If the crowdsale wasn't successful, all the money fromVault is returned to the investors automatically. Once the funds areraised through the crowdsale, the “Production” stage of the project islaunched. From this point, all received funds are stored in “Vault” anddo not go directly to the entrepreneur. After successful completion ofthe crowdsale, the entrepreneur receives funds for the first step in thecontract.

Type SPONSORSHIP DEBT REV. SHARE Description A stakeholder Anentrepreneur A stakeholder buys provides borrows money portions of an IPin resource/support from a lender exchange for to the venture secured bythe tokens and revenue by buying IP to be paid share in the project.portions of the back in with IP in exchange interest for tokens. VaultYes Yes Yes (smart contract money storage) Production Yes Yes Yescontrol (Voting) Bucket — Yes Yes (storage for return) Return — Yes, %Yes, % Return type — Predetermined Lifetime recurrent recurrent payments(e.g. payments (e.g. monthly, weekly monthly, weekly payments) paidpayments) using tokens Business Control — — — (decision making usingvoting system)

If the entrepreneur uses the Revenue Share approach, the token may be asecurity and SEC registration may be needed. If the entrepreneur wishesto avoid security registration requirement, s/he can tokenize the assetand sell tokens to fund the project or borrow based on the asset. Buyersof such tokenized IP can have a use license, among others. The smartcontract dictates the terms for the project's development. The terms ofthe offering are embedded in the smart contract. Upon successfullycompleting each term, the smart contract automatically moves stakeholdermoney as further financing. Stakeholders vote on whether or not the stepwas completed if “YES”, the entrepreneur automatically receives the nextportion of funds and otherwise unused funds from Vault are returned toinvestors. The project may or may not have a Bucket to store fundsearned along the way. For the Revenue Share, and Debt smart contracts,the entrepreneur must deduct a specified amount of money into thecontract's “bucket”. Once the business begins to profit, the funds fromthe bucket are distributed amongst token holders. The accrual of“Revenue Share” and “Debt” contracts is extracted from the bucket at themonthly or weekly payments intervals. The smart contract can be Ethereumbased or any suitable cryptocurrency. In another embodiment, instead ofinvesting in the asset, the investor can invest in the entrepreneurhimself/herself.

In one embodiment, as detailed in Publication 20130317994 to the instantinventor, the content of which is incorporated by reference, theautomated text generation can be used in the framework for converting anidea to tangible asset. An exemplary creative work disclosure view orform receives a title, which is descriptive of the creative work, andshould be less than 500 characters. As the user enters text into thetitle, the software automatically retrieves potentially interesting orsimilar creative works, references, or publications in the right column.The user can click on each document, and a PDF file for the documentwill be saved in a project directory and opened for his/her review. Theuser can use these documents as writing samples, and to see if they arerelevant to the user's creative work and if so he/she can focus text anddiagrams to focus or emphasize aspects that will differentiate thecreative work from the publication or reference. In case the user is notfamiliar with the format of creative work documents, the PDF documentwill provide the user with example writing style to follow. The user maywant to review the results to see if others have thought of the samecreative work already and if so, the user may want to abandon the filingeffort. Seeing how similar concepts are described in professionallywritten creative works may also be helpful to the user in drafting hisor her own text and can be an excellent way to learn how creative worksare written. Relevant documents are then saved for citing to thegovernment agency when required. The Background section is a briefdescription of the issues or problems to be solved by the creative work.It sets up the need for the creative work. The user may want to describeexisting solutions to these problems. In one implementation, aspell-checker is used to highlight potential errors. Instead of focusingon the shortcomings of existing solutions in the Background section, itmay be advantageous to focus on how the user's creative work is superiorto existing solution(s), and such description should be positivelyrecited as advantages of the preferred embodiment in the Summarysection. The Summary section captures what the user considers to be thehighlights of the creative work. The user should describe the creativework at a high level and reference only essential components or elementsmaking up the creative work. Non-essential or optional elements shouldbe described later in the Detailed Description section along with thedrawings illustrating their relationships to the essential elements.Alternatively, the user can list the optional elements in a separateparagraph that begins with an introductory sentence such as“Implementations of the system can include one or more of thefollowing”. The Background section thus collects basic information aboutthe creative work. The user can enter the title or name of the creativework in the first text region. In the second text region, the user canenter background information relating to the creative work, and in thethird text region, the user can enter a brief summary of the creativework.

In editing a figure and describing the figure, a series of drawings isprepared that illustrate the operation of exact embodiments orimplementations of the user's creative work. The user will also want togenerate drawings for alternative ways to implement the creative work toprevent others from designing-around the implementation of the creativework. The drawings can be done by hand and digitized using a scanner ora camera. Alternatively, the user can generate these diagrams usingtools such as PowerPoint and Visio, among others, and import them ordirectly import images (jpeg or png). If the creative work cannot beillustrated, but can be described using photographs, the user can takepictures of the creative work and include them in the application. Thepicture can be taken from a standard digital camera or can be done usinga tablet or cell phone's camera. After the images have been generated,easy to use tools help users import or capture drawings and describethese drawings in detail. The user can annotate these drawings withnumbers up front or can use a tool to place or mark reference numeralsfor elements. Tools are provided to help the user easily and quicklyindicate element names and check consistency of usage from his or hertext entry. Thumbnails of each drawing are shown in a top bar. When theuser clicks on the image thumbnail, an enlarged figure is shown on thebottom left, while text associated with the figure is shown on the rightcolumn.

For example, in one implementation, the user can annotate the drawing byclicking a pointer such as a mouse pointer near the desired area. A boxwith a red dot appears. The red dot is the tip of a pointer arrow. Theuser can drag the red dot to point it to the correct spot in the figurewhere the user wants the text to be associated with. The user can alsoselect the number and move the number to a desired position. The box hasan automatically generated number that can be changed. Further, the usercan add descriptive text after the number to provide more information.If the user enters text after the number, a colon will be shown toseparate the text from the number. During printing of the figure, thedescriptive text after the number will be suppressed. When the userclicks a mouse pointer near the desired area, a box with a red dotappears. The red dot is the tip of a pointer arrow. The user can dragthe red dot to point it to the correct spot in the figure where the userwants the text to be associated. The user can also select the number andmove the number to a desired position. The box has an automaticallygenerated number that can be changed. Further, the user can adddescriptive text after the number to provide more information. If theuser enters text after the number, a colon will be shown to separate thetext from the number. During printing of the figure, the descriptivetext after the number will be suppressed. When the user clicks on thepointer or the element number, the number is shown, along with a red dotindicating the tip of the pointer. The user can drag the red dot topoint to a desired end target on the drawing. The user can also drag thenumber to a desired beginning target.

In an exemplary process to generate the detailed description of thecreative work, the process gets the initial disclosure as well as nounphrases and requested exclusivity elements. Next, for each reference ornoun phrase, the process automatically suggests text for use oralternatively asks the user to provide more details on the element andon the relationship with other elements. The process prompts the user toassociate a number with a noun phrase. In one embodiment, the nounphrase is automatically generated for the user and the user canedit/add/delete the number as desired. In another embodiment, the usercan directly assign a number to an element. The process then asks theuser to generate drawing(s) illustrating the part number on thedrawing(s).

Pseudo-code for performing noun phrase detection in a claim is:

Initialize pointer to current text position in claim. If claim is methodor process claim:  Identify all gerund phrases in claim except“comprising/including/  having” as antecedent basis candidates. Savenoun phrase in an  element table for the claim. Else repeat until end ofclaim  Identify next occurrence of “the” or “said” and identifypotential  end of noun phrase. Save noun phrase in an element table forthe  claim. If noun phrase is gerund + means form, add an entry to cover the “means for” + gerund form.  Look for matching occurrence of “a” or“an” indicating a start of  noun phrase and identify potential end ofnoun phrase. Save noun  phrase in an element table for the claim. Ifnoun phrase is gerund +  means form, add an entry to cover the “meansfor” + gerund form.  If no matching occurrence for current claim textand if claim is a  dependent claim, search parent claims for antecedentbasis.  If all claims have been searched and no match exists, highlight element with “the” or “said” and flag antecedent basis error.  Updatepointer to current position. End if

-   -   The process then generates a draft detailed description of the        creative work for the government application. The draft        description is simply an organized document containing the        descriptions of the elements and noun phrases in a logical        order. This can be done by following the claim sequencing in one        embodiment. Alternatively, the system can follow the sequence        provided in the creative work disclosure and insert the        additional text with the reference and reference numerals        according to the sequence of the disclosure. In other        embodiments, the process can generate text based on a        predetermined order of the reference numerals. The process        allows the user to review, revise and edit the draft description        as appropriate. Once the user accepts the draft, the process        saves the description text as the final version.

To create money to support the students and the educationalinstitutions, the system can monetize the Creative assets by sellingand/or licensing the Creative assets. The system can auction onecreative work at a time on an auction site similar to ebay, for example.In such as system, interested parties bid on the asset and the highestbidder is awarded the asset.

One embodiment provides the ability to provide a pricing requestassociated with a single Creative asset. The method includes receivingone or more orders for each IP listing symbol, selecting an order fromthe one or more orders with consideration of available volume,associated prices, and applying the process to a portfolio of assetswherein an optimized asset pricing across the portfolio is presented tothe seller in association with the request to optimize the pricing of asingle or multiple asset order in a form which includes, but is notlimited to, a list of symbols and associated transaction size and price.

Unique to a market for structured IP products is the ability to listsingle assets for sale under multiple listing symbols, in accordancewith an embodiment of the present creative work. Such work can be tiedto a blockchain entry. The ability to list a portfolio of single assetsrepresented by a plurality of technology codes or listing symbolsenables sellers and buyers to maximize revenue generated by the sale ofthe assets or minimize the expense generated by the purchase of theassets, respectively. In the improved systems, methods, and computerprogram products, a seller of the structured Creative assets requeststhe technology codes or listing symbols, which would represent theoptimal price for the possible sale of an asset or portfolio of assets.In a variation, a buyer is provided with the optimal technology code orlisting symbol or symbols for the purchase of an asset or portfolio ofassets.

Another embodiment includes receiving, by one or more computing devices,an inventory description of an Creative asset for sale; generating, bythe one or more computing devices, plurality of sell orders eachrepresenting the Creative asset using a different combination ofattributes of the Creative asset, wherein the attributes representcharacteristics of technology; prioritizing, by the one or morecomputing devices the plurality of sell orders to generate an orderposting subset of the plurality of sell orders in accordance withhistorical data associated with the plurality of sell orders; andposting, by the one or more computing devices, the Creative asset forsale under each of the sell orders of the order posting subsetconcurrently, wherein each of the sell orders of the of the orderposting subset is usable to match a buy order with the Creative asset,and wherein matching the buy order with any one of the sell orders ofthe order posting subset cancels the remaining of the sell orders of theorder posting subset.

The system can post the asset for sale under each of the sell orders ofthe order posting subset concurrently comprises posting the Creativeasset for sale using a plurality of listing symbols assigned to theCreative asset.

The organized electronic marketplace will be referred to by severalnames throughout this disclosure, including by reference to componentssuch as a brokerage system. One skilled in the relevant arts willappreciate that behavior attributed to any of these components can beallocated to different components of the overall system while achievingthe same desired effect.

The process begins where the creative asset company makes availableaccess rights and audience profile access rights within the organizedelectronic market for structured Creative assets queries third partypublisher traffic/volume data (e.g., analytic data) to establish itsfuture capacity to create commercialization opportunities and audienceprofile access opportunities.

The creative asset Buyer provides viewer profile data and displayscreative asset inventory data to the brokerage system, in accordancewith an embodiment of the creative work. With the necessary data forverification of the structured Creative assets available, the brokeragesystem then cleanses the data, in accordance with an embodiment of thepresent creative work. In particular, the data from various supportedexternal third-party analytics providers (e.g., Thomson, Lexis, amongothers) is reviewed to delete anomalies in the data likely to representerrors or non-conforming asset structures.

Once the verification data has been imported and cleansed, the data canbe reviewed for approval, in accordance with an embodiment of thepresent creative work. With the data approved, it is possible to createand allocate assets into the creative asset Buyer's account with thebrokerage system. The profile data and the traffic/volume data arecombined, and the system then parses the data at step 106 in preparationto post the assets to the publisher's account (IP Producer's account),in accordance with an embodiment of the present creative work. Parsingthe data allows the brokerage system to account for a variety ofthird-party analytics providers, each having their own unique formatsfor publishing analytics data.

In accordance with a further embodiment of the present creative work, ifthe inventory to be offered by the Audience Producer represents onlinedisplay advertising inventory, then the brokerage system checks for thevalidity of the display space inventory by querying the location of thedisplay space. If the system finds that the descriptive display spacedata does not match the results of the query, the system will assign anexpired or error status to the display space access right and remove theinventory from the Audience Producer's account. When the brokeragesystem finds that the descriptive display space data matches what isqueried, or alternatively when the media is not callable (e.g., offlineassets, although one skilled in the relevant arts will appreciate thatthese techniques can be converted for application to other non-webassets), the inventory is processed through the symbology assignmentprocess, which is described in further detail below, in accordance withan embodiment of the present creative work. Once symbols have beenassigned to the asset, the Audience Producer instructs the brokeragesystem to offer its inventory for sale at step 109, in accordance withan embodiment of the present creative work. In particular, once theAudience Producer's inventory has been verified, had symbols assigned,and posted to the Audience Producer's account, the Audience Producer canthen sell or otherwise trade on their inventory. By way of non-limitingexample, the Audience Producer can trade their inventory through theoffer posting process and the bid/offer matching process.

In the above example, the brokerage system is configured to use the dataimported to estimate the number of creative workspace access rights andaudience profile access rights that will be available for advertisingplacement during a broadcast. The creative asset Producer additionallyimports publishers display space inventory data, which describes theattributes of the display space made available by the creative assetProducer. The creative workspace descriptive data and the audienceprofile data are then cleansed where anomalies in the data likely torepresent errors or non-conforming asset structures may be discoveredand removed.

With the data cleansed, the brokerage system approves the data forprocessing. The brokerage system then parses the data to assignindividual display space attributes to a creative workspace access rightasset and audience profile attribute to an audience profile access rightasset, in accordance with an embodiment of the present creative work.

For creative workspace access rights, the system may, subject to certainparameters, automatically verify the presence and characteristic of thecreative workspace. If the system finds that the descriptive creativeworkspace data does not match what is queried by the buyer, the systemwill assign an expired or error status to the access right and removethe inventory from the creative asset Producer's account. If everythingis in order, the assets are provided with symbology through thesymbology assignment process. The assets are then posted to the creativeasset Producer's (publisher) account. Once the assets are in the accountthe creative asset Producer (e.g., inventor) may offer inventory forsale from the assets in the account, using a user interface provided bythe brokerage system. With an offer to sell in place, the offer isprocessed through the offer posting process and then the bid/offermatching process to initiate the transaction in accordance with anembodiment of the present creative work.

One embodiment takes an entity's portfolio of assets and groups theminto assets that can be subject to a floating privilege and those thatthe entity does not make available to others. The portfolio of assetsrepresents the total set of assets the entity owns that could be subjectto transfer to another. The entity, also referred to as the assetportfolio owner, or portfolio owner, is a corporation and assets areassets and creative works in particular. It will be understood that theassets are not limited to creative work or assets, but can be othertypes of assets in which rights can be transferred to others. Theportfolio owner might own thousands of creative works (P). In thisexample, the creative asset owner owns “n” creative works, where n is apositive integer. The creative asset owner, in this example, hasdesignated “m” of its creative works P1 to Pm as eligible for selectionfor transfer to the holders of a floating privilege upon the occurrenceof a predetermined event, thereby forming a floating privilege pool,where m is a positive integer less than n. This floating privilege poolis also referred to as a dynamic asset pool, or more simply a “pool”. Inthis example, three of the creative asset owner's clients, A, B and C,each have purchased a floating privilege for the assets in the floatingprivilege pool 101. The creative asset owner's remaining creative works,Pm+1 to Pn, are not included in the floating privilege pool, but ratherare held by the creative asset owner for its own exclusive use. Althoughcreative works P1 to Pm are shown in the floating privilege pool, thespecific creative works in the pool may change due to the dynamic natureof the pool. The portfolio owner's portfolio of assets 100 can bedivided into a floating privilege pool containing creative works P1 toPm and also contains a custom floating privilege pool. The customfloating privilege pool contains creative works Pm+1 to Pp, where p is apositive integer greater than m and less than n. In this embodimentclient D has contracted with the creative asset owner to have a floatingprivilege to the custom floating privilege pool. Custom pool is adynamic asset pool in that the creative works within the pool can changeover time. Alternatively, client D can agree with the creative assetowner to limit the changes to the custom pool. For example, the contractbetween the creative asset owner and client D can specify that thecertain creative works remain in the custom pool while others maychange. The portfolio owner's portfolio of assets also divided into afloating privilege pool 101 containing creative works P1 to Pm andcontaining a custom floating privilege pool. The remaining creativeworks in the portfolio consist of creative works Pp+1 to Pn. The customfloating privilege pool 303 contains creative works that also areincluded in the floating privilege pool. Accordingly, the customfloating privilege pool contains creative works Pm-q to Pp, where q is apositive integer less than m and p is greater than m and less than n.Creative works Pm-q to Pm are common to both the floating privilege pooland the custom floating privilege pool since they are contained in bothpools. Accordingly, clients A, B, C and D each have a floating privilegefor the common assets in the two pools.

Due to the diversity of the creative work portfolio, the creative assetcompany can extract value from a portfolio of assets, for examplecreative works, utilizing a floating privilege, for example a floatingassignment privilege, is summarized by granting to a client forconsideration by an asset portfolio owner a floating privilege to adynamic set of assets, such as a set of creative works, wherein thefloating privilege is a right to obtain an interest in one or more ofthe assets in the dynamic set upon the occurrence of a predeterminedevent. By employing the techniques described here, a client that lacks alarge asset portfolio can have access to the creative asset collectiveowner's portfolio in a time of need. The client rights to these assetscould be publicized so that anyone considering suing the client wouldhave to consider all of the assets at the client's disposal forcounterclaims. In this way, a floating privilege to a dynamic asset poolprovides both a deterrent value and an enhanced ability for the clientto fend off such lawsuits. The right, or option, is not for any specificasset, since the set of assets is considered to be dynamic following theestablishment of a floating privilege. The set or pool of assets isdynamic because no particular asset is guaranteed to exist at a latertime. The right to obtain an interest in one or more of the assets is aprivilege that is not tied to any particular asset in the pool ofassets, but rather floats over the assets so that it can be applied toany of the assets in the pool. When the privilege is executed, aninterest is obtained to one or more assets selected from the presentlyavailable assets within the dynamic set of assets at the time theprivilege is executed. By agreement, while the set of assets covered bythe floating privilege is dynamic, the number of assets in the set istypically constrained in some way to ensure continuing value to theclient.

An example of such a predetermined event can be the initiation of acreative work infringement action by a third party against the client.The client can then use the floating privilege to select one or morecreative works from among the set of assets associated with theprivilege to assert against the third party. Executing the privilege caninclude granting sufficient rights in the selected creative works togive the client standing to sue the third party for infringement ofthose creative works. For example the client can be granted an exclusivelicense in the creative works or the selected creative works can beassigned to the client. A predetermined event, as used within thecontext of this specification for the purpose of executing a floatingprivilege, may be referred to herein as a “trigger event”. Thus, clientsA-D can buy creative work infringement insurance from the creative workpool owner who in turn uses the money to pay at least a portion of thestudents/educational institution.

The system can operate with edge computing—moving computation and datastorage away from large centers and relying more heavily on localstorage and caching, with reduced energy footprints. One embodimentrelies on transformers in the cellular base stations located near amobile device (such as transformers running on a 5G base station incommunication with a mobile device).

Embodiments are described herein with reference to flowchartillustrations and/or block diagrams of methods, apparatus (systems), andcomputer program products according to embodiments. It will beunderstood that each block of the flowchart illustrations and/or blockdiagrams, and combinations of blocks in the flowchart illustrationsand/or block diagrams, can be implemented by computer readable programinstructions. Computer readable program instructions described hereincan be stored in memory or downloaded to respective computing/processingdevices from a computer readable storage medium or to an externalcomputer or external storage device via a network, for example, theInternet, a local area network, a wide area network and/or a wirelessnetwork. The network may comprise copper transmission cables, opticaltransmission fibers, wireless transmission, routers, firewalls,switches, gateway computers and/or edge servers. A network adapter cardor network interface in each computing/processing device receivescomputer readable program instructions from the network and forwards thecomputer readable program instructions for storage in a computerreadable storage medium within the respective computing/processingdevice. The computer readable storage medium can be a tangible devicethat can retain and store instructions for use by an instructionexecution device. The computer readable storage medium may be, forexample, but is not limited to, an electronic storage device, a magneticstorage device, an optical storage device, an electromagnetic storagedevice, a semiconductor storage device, or any suitable combination ofthe foregoing. Computer readable program instructions for carrying outoperations may be assembler instructions, instruction-set-architecture(ISA) instructions, machine instructions, machine dependentinstructions, microcode, firmware instructions, state-setting data,configuration data for integrated circuitry, or either source code orobject code written in any combination of one or more programminglanguages, including an object oriented programming language such asSmalltalk, C++, or the like, and procedural programming languages, suchas the “C” programming language or similar programming languages. Pythonhas a large amount of libraries that are super handy for implementingsentiment analysis or machine learning from scratch. NLTK, or theNatural Language Toolkit, is one of the leading libraries for buildingNatural Language Processing (NLP) models, thus making it a top solutionfor sentiment analysis. It provides useful tools and algorithms such astokenizing, part-of-speech tagging, stemming, and named entityrecognition. SpaCy is an industrial-strength NLP library in Python whichcan be used for building a model for sentiment analysis. It providesinteresting functionalities such as named entity recognition,part-of-speech tagging, dependency parsing, and word vectors, along withkey features such as deep learning integration and convolutional neuralnetwork models for several languages. Scikit-learn is a machine learningtoolkit for Python that is excellent for data analysis. It featuresclassification, regression, and clustering algorithms. TensorFlow is thedominant framework for machine learning in the industry. It has acomprehensive ecosystem of tools, libraries, and community resourcesthat lets developers implement state-of-the-art machine learning models.PyTorch is another popular machine learning framework that is mostlyused for computer vision and natural language processing applications.Developers love PyTorch because of its simplicity; it's very pythonicand integrates really easily with the rest of the Python ecosystem.PyTorch also offers a great API, which is easier to use and betterdesigned than TensorFlow's API. Keras is a neural network librarywritten in Python that is used to build and train deep learning models.It is used for prototyping, advanced research, and production. CoreNLPis Stanford's proprietary NLP toolkit written in Java with APIs for allmajor programming languages to extract the base of words, recognizeparts of speech, normalize numeric quantities, mark up the structure ofsentences, indicate noun phrases and sentiment, extract quotes, and muchmore. OpenNLP is an Apache toolkit designed to process natural languagetext with machine learning and supports language detection,tokenization, sentence segmentation, part-of-speech tagging, namedentity extraction, chunking, parsing, and conference resolution. Weka iscomprised of a set of machine learning algorithms for data mining tasks.It includes tools for data preparation, classification, regression,clustering, association rules mining, and visualization. R is aprogramming language that is mainly used for statistical computing. Itsmost common users include statisticians and data miners looking todevelop data analysis. Caret package includes a set of functions thatstreamline the process of creating predictive models. It contains toolsfor data splitting, pre-processing, feature selection, model tuning viaresampling, and variable importance estimation. Mir is a framework thatprovides the infrastructure for methods such as classification,regression, and survival analysis, as well as unsupervised methods suchas clustering.

The computer readable program instructions may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider). In some embodiments, electronic circuitry including, forexample, programmable logic circuitry, field-programmable gate arrays(FPGA), or programmable logic arrays (PLA) may execute the computerreadable program instructions by utilizing state information of thecomputer readable program instructions to personalize the electroniccircuitry, in order to perform embodiments.

Additionally, it is understood in advance that the teachings recitedherein are not limited to a particular computing environment. Rather,embodiments are capable of being implemented in conjunction with anytype of computing environment now known or later developed. For example,cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (for example, networks, network bandwidth, servers,processing, memory, storage, applications, virtual machines, andservices) that can be rapidly provisioned and released with minimalmanagement effort or interaction with a provider of the service. Thesoftware/system may be offered based the following service models:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (for example, web-basede-mail). The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (for example, host firewalls).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes. Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(for example, mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (for example, cloud burstingfor load-balancing between clouds).

Unless otherwise explicitly stated, articles such as “a” or “an” shouldgenerally be interpreted to include one or more described items.Accordingly, phrases such as “a device configured to” are intended toinclude one or more recited devices. Such one or more recited devicescan also be collectively configured to carry out the stated recitations.For example, “a processor configured to carry out recitations A, B andC” can include a first processor configured to carry out recitation Aworking in conjunction with a second processor configured to carry outrecitations B and C.

As used herein, the terms “determine” or “determining” encompass a widevariety of actions. For example, “determining” may include calculating,computing, processing, deriving, looking up (e.g., looking up in atable, a database or another data structure), ascertaining and the like.Also, “determining” may include receiving (e.g., receiving information),accessing (e.g., accessing data in a memory) and the like. Also,“determining” may include resolving, selecting, choosing, establishing,and the like.

As used herein, the term “selectively” or “selective” may encompass awide variety of actions. For example, a “selective” process may includedetermining one option from multiple options. A “selective” process mayinclude one or more of: dynamically determined inputs, preconfiguredinputs, or user-initiated inputs for making the determination. In someimplementations, an n-input switch may be included to provide selectivefunctionality where n is the number of inputs used to make theselection.

As used herein, the terms “provide” or “providing” encompass a widevariety of actions. For example, “providing” may include storing a valuein a location for subsequent retrieval, transmitting a value directly tothe recipient, transmitting or storing a reference to a value, and thelike. “Providing” may also include encoding, decoding, encrypting,decrypting, validating, verifying, and the like.

As used herein, the term “message” encompasses a wide variety of formatsfor communicating (e.g., transmitting or receiving) information. Amessage may include a machine readable aggregation of information suchas an XML document, fixed field message, comma separated message, or thelike. A message may, in some implementations, include a signal utilizedto transmit one or more representations of the information. Whilerecited in the singular, it will be understood that a message may becomposed, transmitted, stored, received, etc. in multiple parts.

As used herein a “user interface” (also referred to as an interactiveuser interface, a graphical user interface or a UI) may refer to anetwork based interface including data fields and/or other controls forreceiving input signals or providing electronic information and/or forproviding information to the user in response to any received inputsignals. A UI may be implemented in whole or in part using technologiessuch as hyper-text mark-up language (HTML), ADOBE® FLASH®, JAVA®,MICROSOFT®.NET®, web services, and rich site summary (RSS). In someimplementations, a UI may be included in a stand-alone client (forexample, thick client, fat client) configured to communicate (e.g., sendor receive data) in accordance with one or more of the aspectsdescribed.

While the above detailed description has shown, described, and pointedout novel features as applied to various embodiments, it can beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. As can berecognized, certain embodiments described herein can be embodied withina form that does not provide all of the features and benefits set forthherein, as some features can be used or practiced separately fromothers. The scope of certain embodiments disclosed herein is indicatedby the appended claims or requested exclusivity rather than by theforegoing description. All changes that come within the meaning andrange of equivalency of the requested exclusivity are to be embracedwithin their scope.

What is claimed is:
 1. A method to generate a document, comprising:providing a document structure having one or more first text prompts, asecond text prompt, and a plurality of component texts; from the one ormore first text prompts or second text prompt, generating one or moreartificial intelligence context-sensitive text suggestions, by applyinga transformer with an encoder on the one or more first text prompts,second text prompt and the plurality of component texts and a decoderthat produces a text expansion with token biasing to provide theartificial intelligence context-sensitive text suggestions generationbased on the one or more first text prompts, second text prompt, and theplurality of component texts; wherein generating one or more artificialintelligence context-sensitive text suggestions further comprises:applying generative artificial intelligence with pre-norm alization andchangeable tokenization with token biased weights for a zero-shot,one-shot or some-shot environment to generate the artificialintelligence context-sensitive text suggestions from the one or morefirst text prompts, second text prompt and the plurality of componenttexts; and generating a document text by combining the generatedartificial intelligence context-sensitive suggestions, the one or morefirst text prompts, the second text prompt and the plurality ofcomponent texts.
 2. The method of claim 1, wherein the documentstructure comprises an outline, wherein each of the one or more firsttext prompts comprises a chapter or section overview, and wherein theplurality of component texts comprises a chapter or section outlinespecified by a user which is subsequently expanded as the one or moregenerated artificial intelligence context-sensitive text suggestions. 3.The method of claim 2, wherein the document text comprises a fictionwork, a non-fiction work, a computer readable code, a machinespecification, a patent application, or a mechanical description.
 4. Themethod of claim 1, wherein the document structure comprises one or morefigures, wherein each figure comprises a brief description of thedrawing, a figure description overview, and anartificial-intelligence-generated detailed description of the figurefrom component texts corresponding to items in the figure.
 5. The methodof claim 1, wherein token biasing further comprises biasing neuralnetwork weights.
 6. The method of claim 1, wherein the combining furthercomprises combining a title and a background text with the one or morefirst text prompts and providing the combined title, background, and theone or more first text prompts to a learning machine to synthesizeartificial intelligence context-sensitive text suggestions.
 7. Themethod of claim 1, further comprising: extracting one or more referencesfrom a figure and annotating the one or more references with text; andforming one or more artificial-intelligence-generated reference textsuggestions.
 8. The method of claim 1, further comprising performinggrammar analysis and suggesting grammar correction and editing thedocument for conciseness.
 9. The method of claim 1, wherein thetransformer comprises a generative pre- trained transformer (GPT). 10.The method of claim 1, wherein generating the artificial intelligencecontext-sensitive text suggestions further comprises applying GPT(Generative Pre-trained Transformer) model or a BERT (BidirectionalEncoder Representations from Transformers) model.
 11. The method ofclaim 1, further comprising determining when two pieces of text,component, module, code, data structure, or image perform a similar taskand showing the determined text, component, module, code, datastructure, or image to a user.
 12. The method of claim 11, furthercomprising breaking-down the second text prompt into one or morealternate components with different component text but capable ofperforming the second text prompt based on teachings from prior artdocuments and showing the one or more alternate components as anartificial-intelligence-generated design around satisfying the secondtext prompt.
 13. The method of claim 1, further comprising detectingplagiarism in the generated document text by matching the generateddocument text to text crawled from the Internet.
 14. The method of claim1, further comprising generating a part list by detecting noun phrases(NPs) in the generated document text and corresponding numbers for theNPs.
 15. The method of claim 1, wherein the generated document textcomprises a patent application, further comprising generating a list ofclaimed elements.
 16. The method of claim 1, wherein the generateddocument text comprises a patent application, further comprisinggenerating a list of unclaimed elements.
 17. The method of claim 1,wherein the generated document text is part of a portfolio accessible toone or more licensees.
 18. The method of claim 1, further comprisinggranting licensing rights to the document.
 19. The method of claim 1,wherein the generated document text comprises a chatbot text.